DeviceNet is a communication bus technology developed by Allen Bradley PLC systems to interface and control equipment connected through a breaker in the MCC. With it, a host like Allen Bradly PLC or DeltaV can access the parameters of a motor to get information on that motor, eg rpm, load current, temperature, all kinds of neat stuff. It can actually be used as well for on-off control through host configuration, instead of using relays.
I knew absolutely nothing about it when I was asked to go up to Suncor Firebag, north of Fort McMurray, to troubleshoot it. Apparently "something" was shutting down some of their big pumps that were tied to DeviceNet equipment. Our DeltaV control equipment was being blamed.
When I got up there, the plant manager came into the office where I was discussing the issues with two instrumentation techs, Bill Barber and Wes Meger. The manager asked if I was the guy from Spartan and when I answered in the affirmative, he said, "You better get this goddamn system working or you can take it back with you". Hooookkkaaayyyy!
By now, after years of experience with customers and DeltaV, I could take these threats with a grain of salt, because the problem almost was NEVER DeltaV because our system was basically just the "window" into the process. Bill and Wes showed me the configuration and how they tried different ways of doing things, but still the problem persisted. While I was there taking my safety indoctrination, portions of the plant shut down 3 times when the 800 horsepower boiler feedwater pumps suddenly stopped working!
Not knowing anything about specific equipment or process puts me in a good position because the first thing I have to do is study every facet about it, taking nothing for granted. An expert on the other hand could make some incorrect assumptions, missing the problem in the first place. I love troubleshooting new stuff.
My plan of attack was first to see if the DeviceNet was installed properly and with my new DeviceNet meter, DVM, flashlight and note book, I went out into the plant to investigate the entire network with Bill and Wes. We started at the DeltaV node where the interface was to Devicenet, checking the wiring for proper connections, proper shielding and of course, the termination resistor. I measured all cable drops and trunk lengths, then compared it with the specs from Allen Bradley. We did this for the entire plant and this plant was HUGE. Just stepping out the trunk cable between two nodes took me 1/2 hour as it convoluted its way up and around cable tray for about 1000 feet! All lengths, number of nodes, cable type and size was correct, although we did find a number of termination resistors improperly applied: no resistors, 1 resistor, 3 resistors, all combos including the correct two resistors, one at each end.
By determining what was failing and actually being in the MCC when one of the devices stopped working, we were able to narrow down the search to this one building, then to one drop. Eventually I took out my trusty DVM and did a continuity check between the trunk entry to a large cable tap and 8 drops to equipment breaker panels and found a break internal to the tap. We replaced it and solved the problem. I took the tap home and was cutting it up and trying to dissolve the hard epoxy to get to the fault when the manufacturer, Woodhead, asked me for the materials to do their checks. This tap only contained bus bars from the trunk connector of 5 pins to the 8 drops and one of the connections had a bad solder joint. The tap was worth about $100 and yet caused hundreds of thousands of dollars in lost production and employee time.
It wasn't the fault of DeltaV.