[The following is a lightly edited transcript of a webinar.]

Carl Dyke: Mark and I have come and gone over the years from a good number of tricky scenarios where things weren’t working well in hydraulics and we always find at every single occurrence is a real learning opportunity, both for those who own that machinery and for ourselves. It never seems to end that the process of troubleshooting a system, Mark, would you agree, is a humbling one. You have to really keep your mind open and learn to observe and follow some kind of a process other than guesswork, right, Mark?

Mark P.: Yes, I would say that the biggest learning experience I’ve had over my career is getting to the age where you start to be humble and you say “I know what I’ve learned, but I need to listen to the people that are trying to explain their problems.” And then as they talk, I try and apply it to what methodical process has worked for me over the years and it starts with listening, it starts with defining problems. And once we do that, we can start attacking it.

Carl Dyke: Yes, so there’s an endless number of different styles of hydraulic circuits out there. In our program today, we’re not going to try to assert exactly where one should insert a flow meter or a pressure gauge. The possibilities are so many. We’re going to focus more on plans and methods to put to work in your troubleshooting activities when they occur. That help kp it organized. And Mark is a big stickler for planning and getting ready. So I know he’s gonna have a lot to say about that. eMark is forever cataloging data for our client systems and making sure that when a problem crops up, that there is a readiness. You’ll also find out that Mark’s very passionate about keeping solid contaminates out of his system. So that will pop up as well. Other webinars that Mark and I have done, we’ve filled the slides with images of things that have failed and isn’t this interesting how this contaminate wedged into this valve or that valve … For this webinar, we really wanna look at a business method, if you will, for helping companies with troubleshooting.

\So just to get it underway, let’s just go back to the beginning and think for a moment about why hydraulic system breakdowns happen in the first place. And trying not to overlook basic causes. One of the key things that gets in the way with keeping hydraulic systems running well is simply the way we operate them. We’ve got issues where operators are working a piece of machinery beyond its capability. That’s something that can crop up more easily, I suppose, in mobile machinery. That can happen in the plant or mill environment as well. And if you’re looking at the slide, you can see some of the solutions for those things are fairly basic, but they have to occur. Some manufacturers are very concerned that you’ll damage their systems right out of the starting gate, if you didn’t read and carefully follow a break in period, or some kind of commissioning process that they have in mind.

Mark P.: We’ve come across it so many times when you actually approach a machine that you haven’t seen before, from an experience point of view and say “Let me just read the book real quick,” and you see some things stated which seem to be obvious to somebody who’s in the hydraulic industry. They would see it and say “Well, did anybody write this down? Did anybody say that they did this? Can you verify that it was flushed? Can you verify that you did a contamination control system right from the get go or did you actually just hope to God that it’d work one day?” And I’ve seen it over the years that we just get back to the basics. It always seems to come back to simple things that are not done because people think they’re not important.

Carl Dyke: It is also possible that breakdowns may happen due to a design weakness. Mark and I find that this is really quite common in the one-off types of machines. Quite often we encounter, in the oilfield more than anywhere else, we encounter one-offs. Things that were only designed one or two times and perhaps the engineering wasn’t quite as complete or refined as it should have been. And so there’s an inherent weakness there and so breakdowns will happen. But even on some of the big brand names. Even some with yellow paint, occasionally, there’s a component in the system where the pressure rating of that component was perhaps a bit too optimistic and …

Mark P.: Yeah, I’ve come across many times where when somebody says “Well, this pump is good for 2,500 psi,” and I’ve come from a bit of an engineering background where we’ve always had to support any statement made like that, and you start reading the data sheet and find out that okay, it is maximum pressure rated for 2,500, but then you’ll see a little asterisk somewhere, or a footnote, and you look down at the bottom of the data sheet and you’ll find out that it’s only good if you run it one percent of its duty cycle time at that pressure. So many times when I’ve seen systems, especially like Carl said, in the one-off situations, where people have just said “Oh, yeah. That’s good. Let’s just design it that way, and it’s good for 2,500.” Well, it is if you only run it 1% of the time.

Carl Dyke: Poor maintenance is certainly a big factor and in some cases, it could be something as simple as a machine slide or some kind of a swing bearing that was not greased, not lubricated, and then the next thing you know, the hydraulic system pressure has been driven up to maximum, because the resistance of the machine movement is so high, and now someone is troubleshooting why the system is hot, and why is it operating at maximum, not actually a hydraulic system fault, more of a basic machine maintenance fault. So sometimes causes are as simple as a failure to lubricate, a drive chain not being lubricated would be putting exceptional stress and strain on a hydraulic motor. Could be something as simple as electric connections to valves, solenoids, or to pressure sensors that is wet or loose. Components being used past their expected service life. We recently came across some clients who had extremely high cycle counts on cylinders that were now worn out and producing particles internally. They just didn’t keep track of an expected service level.

Mark P.: And the poor maintenance side of things, as well, I’ve been in many situations and covering shift or something myself and be given a PM sheet that is so inadequately designed to check a system, and they’ll have comments like “Check pump” and you watch guys and going through a maintenance departments that are just checking boxes, “Yep, the pump is still there.” They do nothing or try to do any kind of preventative maintenance or strategy based on actual performance of the pump. So over the last years, it was certainly something that you started to focus on, going when you say “Check the pump,” well check the pump for its flow and pressure capability. And start to get into objective standards that can be met, as opposed to subjective opinions of “Yeah, that looks good” or “The paint’s still yellow.” We had one customer who said “You can always tell when the pump’s going bad because the color changes.”

Carl Dyke: Wow. Well, yeah. So a lot of failures are simply failure to pay attention for opportunities that you have during the maintenance of your system. Are hydraulic system breakdowns inevitable? Where the movement for reliability-centered maintenance is very strong and there’s a lot of science behind things that should be done that are proactive, condition-based monitoring philosophies, which are all very good and important and companies should pursue these. But reliability-centered maintenance practitioners themselves agree that a certain percentage of equipment breakdowns will still occur randomly, or at least what appears to be very randomly, even with these best maintenance practices in place. So it is absolutely best to be prepared for the eventuality.

Mark P.: And statistical analysis, I know in working with that over years ago, just understanding that if you run something a number of times, there is going to be a failure. And those are not the ones we’re talking about here today. We’re talking about the ones that continually repeat themselves. Put yourself into a cycle of re-occurrence and we’re looking a lot today at how to break the cycle.

Carl Dyke: Yes, so we’ll get to that further on, how to break that cycle. What typically happens when the system breaks down? Well, in short, there’s financial impact. And a lot of chaos is difficult for environments that Mark and I come and go from. Sometimes it’s a quick move to start replacing components in the system. And again, all of those things basically boil down to a cost to the business that can be minimized if troubleshooting that is prepared for and organized.

Mark P.: This is something that we find in business processes, some teams worked out in advance, if you start thinking about them, let’s set some goals. In many companies where they set specific goals, and then you walk by a chart two months later and you see the graph going downhill and you ask somebody “Well, what’s going on now? What’s being done?” “I don’t know. We’re just losing money. We need to do something.” And you find that there’s … It’s not what somebody expected. Whoever created the maintenance goal was not involved in the what happens if it doesn’t mode. We’re really looking at how the culture in the business is on the maintenance side, for sure.

Carl Dyke: We’re looking at response systems and diagnostic routines. Let’s dig into that first one and let’s look at how we can get ready. First of all, keeping documentation correct and current, Mark. That’s a thing that’s next to religion for you.

Mark P.: Yes, next to contamination. It is something that I have … Many companies that we work with, they have no idea what was supposed to be there. They have no idea. They don’t have a reason why we now have this changed. You ask somebody “When did we move away?” Because the original manual said this. The documentation crisis, we call it, Carl and I both say, this is what’s really killing us today. We’re in an information age, and we go one extreme to the other. Either people have no documentation, or they have so much that nobody’s got time to look at it to see what it’s telling you. The guy on the floor really has to be paying attention, as well.

Carl Dyke: Mark, the manufacturer may have their own specifications and list of key pressures and what have you, for a machine. We often encourage our clients also to observe their machine, where there are instruments, and make their own recordings of normal. Write down what those normal pressures are, those temperatures. If it’s cylinders, you may be capturing cycle times. If you have the instruments and can safely do so, you might also be recording the RPMs of hydraulic motors. Keeping track of the normal sounds and vibrations, having all of this data at the ready, really speeds up the process for troubleshooting because it becomes your reference of what normal is.

Mark P.: Yeah, and we run into this all the time with mobile equipment customers who are … I constantly talk to people what’s normal in January is not what’s normal in June. And recently being down in the southern US, I found out what’s normal in January is hot and what’s normal in June is hotter. Where we have, in Canada, the extreme cold and the extreme warm. So we have to know what normal is and when does it happen.

Carl Dyke: We need to have tools available to properly do repair work for when that moment comes. Instruments we find in so many environments we come and go from, are sorely lacking. Quite often, it’s difficult to even find a few pressure gauges that are reliable and prepared for use as insert diagnostic instruments. So very hard to complete the troubleshooting cycle without making sure that you have an adequate number of measuring instruments both for pressure and flow, also for temperature. These are the things that really can’t be overlooked. Additional resources you may need may include polars, or jigs or fixtures. These resources need to be put in place.

So when a fault finally does come in, it’s very easy to jump into a mode of thinking too complex. I know we always have these meetings in business where we’re supposed to keep it simple, but very easy to fault there. Something wrong with our own processors, our brains, where we often jump to complex before we get some basic and simple things done first. Mark and I always encourage that technician should be dispatched, a qualified, trained technician should be dispatched as soon as possible, but his first job is to confirm the fault and find out is it real?

Mark P.: Yeah, well, I’ve come across many, many situations where … And I’m sure everybody has seen that at some point … A pressure gauge that has a magic marker line drawn where pressure should be and then have an operator just totally c

ome unglued when he sees that the gauge is under. And so you have to sit down at some time and explain to them, and again well-trained operators are a big asset to a company, take the time to understand that when the pressure goes down doesn’t necessarily mean we go in and turn pressure up. You have to first understand what pressure comes from, and that would be something that said something just got a lot easier to work than it has normally been. So maybe something is flipping. Maybe something’s letting go, loads change, not getting full loads. Any number of things can be there, so we gotta verify, as Carl said. We gotta verify the fault first, and take control when you have that training and background to say “Wait and minute. Is this really a problem?”

Carl Dyke: That’s great. Absolutely, Mark. And then so from there, I often think of it as … Well, if you’ve been watching CSI Miami and you’ve seen what happens with forensic investigation, now you are CSI machinery. You really do need to adopt some of the methodology of a good forensic investigator who makes sure that he gathers a very careful set of observations and clues and facts and reports from those involved and organize that information. It’s not something that gets done in just one or two minutes. It can take a bit of time to organize that machinery coming in … Sorry, that information coming in from the machinery operator.

Mark P.: Well, my practical experience too, Carl, has been when I go to a machine and I first talk to an operator, I make a point of writing down, in my notebook, whatever, exactly what I’m hearing him say. Then I like to show him “Okay, here’s how I understood what you just said. Can you confirm what I’ve written down?” And that not only helps me understand the situation, but it also helps me confirm that down the road, when the story changes, you can explain to people “Look, I started based on what the operator told me. I wrote it down. Here it is. Joe said this is what he saw.” And it gives us a foundation to build on. And that’s why when we say you have to ask questions again and again and again, because, as Carl said, the CSI machinery concept is that the story … You’re building evidence to create a theory in the end. You’re not creating the theory first and then trying to find evidence to support it. So we gotta make sure we understand the problems that we’re working with before we do anything.

Carl Dyke: So with lots of data gathered from the operator, and initial observations made by the technician, we enter into the diagnostic process itself, the hard thinking work that a technician is going to have to do.

Mark has a lot of clever sayings and one of the early ones he threw at me was “A problem well-defined is half solved.” And of course, the meaning from that is that it’s hard to have too much good quality information to get your diagnostic under control. So to be able to define and to localize and to narrow in and describe, define that problem as carefully as possible, it sounds like it’s something that consumes time, and of course, everybody’s in a hurry when the machine is down. But this has to be done.

Mark P.: And it is just a steady progression of perseverance, of a methodology that you’re comfortable with. I can’t take credit for that saying. I read that somewhere, but it made sense to me that if I knew what was wrong with something, I’d know what to fix. And so when we look at the nature of problems as we’re showing you there, if we could break problems down into these three basic categories, but also you have to constantly train yourself to understand that a pressure valve can cause a flow problem, or a flow valve can cause a pressure problem. But that’s not what we’re looking at right now. Some fundamental things, if the load is moving too fast or too slow, there is a flow problem. Don’t jump to conclusion that it’s a flow valve.

Carl Dyke: Alright, Mark. So you and I are big fans of working on schematics and making sure they’re accurate and teaching technicians have to read them. What are our thoughts about schematic reading, Mark?

Mark P.: Well, in our experience and training, as we move around the countries and we start to come across these guys … We had someone recently at our old office in a training session who said “That thing might as well be written in Chinese for all it means to me.” And I said “Well, you’re just getting way too complicated.” Because the schematic is the best tool you can have. And I’m pretty much lazy and old now. I like to sit in a room with a cup of coffee and analyze what’s going wrong or what could be going wrong before I go out there and start climbing on a machine and getting dirty and getting frustrated. And I can create a scenario here that says let’s methodically go through what could be causing these problems and that schematic is the most vital tool, but it’s only as good as you can read it and it’s only as good as it’s drawn.

Carl Dyke: If you can use that schematic, if you can read it well, you should be able to, using the data from your earlier observations in that schematic, you should now be able to make some kind of a suspect list. Here we are in our CSI mode. You should be able to make some kind of a suspect list and even though you may have quite a few items on that list, from there your plan is to go from easy to complex. You’re getting ready to do the actual hands-on work, but you’re organizing it to make sure that you don’t dive in to something deep and unlikely without first tackling and examining things that are most likely.

Mark P.: We had a situation when I first started in the business of a guy who got flown up to Arctic, only to find out that the level in the tank was down and nobody had checked that. They had spent time and panicking and costing lots of money and they flew him up there. And in the meantime, he gets up there and the storm comes in, the next thing you know, he can’t leave for four days. But all he had to do was somebody check the level of the tank. That simple.

Carl Dyke: Absolutely. So I mean that just plays into where we’re going next here where even though the technician himself now has moved into the diagnostic process, it’s still about observe, observe, and observe. Make sure that you’re working safely. Anyone who is going to work on hydraulic machinery is going to do any adjustments while it’s running live, or is going to do any repairs after it’s shut down. There are hazards. Absolutely you must get hydraulics training before you dive in. We’re going to have to make the assumption that anyone who’s going to get deeply involved with this machine is prepared with safety training.

Mark P.: Knowing where pressure can be stuck, trapped, what if a valve does stick? Don’t rely on automatic systems to bleed down. I came across a situation one time where it was … And I see this often in service manuals, where when you shut the machine down, it should bleed down in 90 seconds. Well, it should and it does, is a whole different thing, or whether it does. You have to confirm and be very confident before you touch the system, as to where pressure is, or is not.

Carl Dyke: Again, taking time to compare the observations, even those the technician is gathering with specifications from the manufacturer. Mark has been spending a lot of time lately working with bale press and slab presses in the pulp mill environment, finding out that they have instruments in the background. There are quite a few pressure sensors and they have been logging data. Wow, that can be very powerful being able to compare what is currently happening at different stages of the machine and compare that to how it was back when the machine was running correctly. So we’re still taking the time to make sure that we’ve gather needed data, that we don’t jump to the wrong conclusion.

Mark P.: And in today’s world, we’re starting to come out with companies like some of the sponsors I see here today who have that kind of sensor documentation, data recording. You can’t go wrong by having a historical reference for what is normal in a machine, so that you can observe okay the only things that can make this go up or this go down, whether it be flow or pressure, there’s some methodical way of looking at it and in essence, everybody who knows the system should be in agreement. You can’t say “Well it can’t be that,” and this guy says it can. We have to come to some agreement that we understand why this could be causing the problem and then we all move towards the same goal. And that’s fundamental for what I work with with groups of people. And especially when we’re in a panicked situation when a machine’s down, everybody just wants something to happen. But be methodical.

Carl Dyke: Absolutely. And so, Mark, is it always contaminants? I mean, we arrive at a machine operator’s location and they tell us that this valve bank must be cracked internally. Well, why is that? Because the boom keeps drifting down. There’s no oil leaking out of the system. If the boom keeps drifting down when it’s not supposed to, this valve manifold, it must be cracked or washed out internally. And what do we usually find?

Mark P.: Oh, we’ve taken many things apart. We’ve been given them as examples of specific valves or something and of course we have the time to take them apart here and try and see what we find wrong and in more cases than none, we find it’s just a stuck spool and it’s all pretty standard teaching for me. And I know Carl uses the same one. When we start to simplistically look at hydraulic systems, if you ask yourself “What happens if this sticks shut?” “What happens if this sticks open?” And “What happens if something sticks halfway in between?” Those are the things that we have to get trained on, understand and look for when we’re thinking things through.

Carl Dyke: When we’re thinking things through as carefully as we possibly can because when we’re all done getting close
to the end of our diagnostic process, in order to verify … That’s where we’re going next here … In order to verify the fault, we may have to do some instrument testing and it may even require us to install a flow meter to find out if there’s leakage through a valve that should be closed during that stage, if there is flow occurring somewhere where it should not be during that stage of the machine cycling. And the reason we’re spending so much time to analyze and plan and think carefully, Mark?

Mark P.: Is the instruments tell you true story. We’re dealing with a medium here that can’t be seen. We’re just making observations of things that are moving and a flow meter … I remember in the old days, this is nice that we moved into a more safety driven culture today here, at least the people that we’re working with, that we used to just what old school people would call “open bucket test it.” Let’s just hold that hose in a bucket and see if oil is coming out. Well, I’ve more than once wore oil because I didn’t understand the difference between a pilot operated valve and a direct actingvalve. And today, we no longer encourage that. We say “Put a flow meter on the line, connect it back to the tank, and watch and observe whether there’s flow coming through here.” Don’t dump anywhere and just think it should be closed. Well, then nothing should be coming out. Take your time.

Carl Dyke: And the other point is that these instrumentation tests, these are tests take time to conduct. It’s gonna take a fair amount of time to gather adapters and fittings and perhaps an extra section of hose to be able to insert that flow meter. So all of this work that we’re doing to plan and think and observe is based on the reality that we may not be given 12 hours to flow meter test a dozen different points. That’s poor diagnostic process in any case. If our process is careful, our observations are careful, we should be at a point where if we have to put a flow meter in it, it might be one, maybe two places. They take time to get down. So the planning and careful observation is so important.

Mark P.: Well, the planning also comes around keeping track of your adapters and hoses that make this process simple and quick. If I have to spend a half and hour looking for a hose or making up a hose, I’m not planning here. I’m just reacting to the situation. So keep track of your stuff and keep it in good working order and clean and stored properly and you’ll get to that end really quick.

Carl Dyke: So the ultimate goal of the diagnostic process, as the technician is working through, is to safely solve the
problem with the minimum amount of resources. That’s the minimal amount of people’s time and the parts and all of that done safely, of course, to return the machine to normal operation. But at the same time, it’s about making sure that there’s some improvements done, whether it’s to the way components are installed or tested, or to the way the machine is operated. Ultimately, we want to try and find a way to avoid the repeat of that same problem.

Mark P.: I used to laugh all the time at people tell me “We have to change that every three months.” Boy, I don’t know about you but I’d get tired of changing the same thing over and over. I could have got a job at an assembly line at that point.

Carl Dyke: Mark, is it possible to find and repair and replace a faulty component and not have solved the fault that was
reported?

Mark P.: Oh, absolutely. I mean, you get into the thing, especially from my last years in the field where contamination is a problem. Contamination doesn’t just go to one component. And so we can end up having multiple things wrong. It’s one of the more fun things we do when we’re teaching troubleshooting on circuitry is to throw in multiple faults that somebody thinks well, it must be this valve. And so you fix that valve and it’s still got the problem. Well then you see the methodical way of doing this is your testing is to verify your observations, what you thought was wrong and will prove that that’s the problem by sticking a flow meter or a pressure gauge in the appropriate place. And then saying “Yep, see, we have flow where we shouldn’t,” or “We have pressure where we shouldn’t,” or “We’re not enough pressure.”

Carl Dyke: So if we found a faulty component, if we made a repair or replacement, one of the key things to do is remain in that diagnostic mode, keep using that systems thinking, and keep serving that internal customer, the machine operator or the operations department, keep supporting them to make sure that machine has been proven to be fully back at its normal operating state.

Mark P.: Well, you made a key point there, Carl, about systems thinking. So often, we find mechanics or millwrights who are focused around a single component and not realizing how that component, when added to multiple components, create a system. And so the reaction of a faulty valve here can cause various problems throughout the system. But when you’re going into what he was just talking about, as far as verifying that this machine is being returned to the field 100% working, then you have to know what that is and you have to prove it an verify it, and test it and then hand it over and give it back into production.

Carl Dyke: Well, let’s have a look at a typical troubleshooting cycle that we run into more often than we wish we did. If we start on the left side of that cycle diagram there, the nine o’clock position, a breakdown occurs. Well, moving through the diagnostic that we’ve described, hopefully the failed parts will be found, perhaps the failed part is now replaced or repaired. And the next thing you know, the machine is back into production and everybody’s happy and walks away from the machine. But so often is the case that the same cycle repeats. One of Mark’s clients would often remark to us that he didn’t really need these kinds of diagnostic procedures. When he called him on the mill radio for the plywood press, he already knew what sun cartridge to go and get because it would be stuck with contaminants. He knew what wrenches to grab and he would head out to the press to replace it. That’s all he needed for troubleshooting was a call on the radio. But of course, he was stuck in that cycle.

Mark P.: Absolutely. Instead of going through the what’s causing that valve to stick all the time, then we’re getting down to what we’re talking about now is get down into the root cause and verify that and break that cycle.

Carl Dyke: So an effective troubleshooting process, really, when we get to the point where we have found a failure, a failed part, we’ve made a repair. Well, that’s great. But if we’re really interested in becoming great troubleshooters, then we’re also interested in doing whatever we can to make that problem go away permanently, or at least semi-permanently, to find some kind of a more lasting solution and break that cycle.
So here comes root cause analysis, and more careful thinking, to figure out what can we do to keep that problem from coming back. The ultimate goal, of course, if we’re in that dark yellow box on the left side, which is so typically the case, we have these run times shown in the green bar and moments times of failure and repeat and repeat. But if we’re conducting the right kind of troubleshooting aimed at improving the system, we should eventually arrive where we are the right, where breakdowns are seldom occurring and perhaps have very short-lived and minimal impact on machine productivity.

Mark P.: And keep in mind too that there is no 100% black and white method for troubleshooting. It’s a methodical process that which you have to think through, re-think, ask a question, re-think, make sure that you’re continually moving towards that end goal of breaking the cycle.

Carl Dyke: Whether or not you have engineering help available to you locally right at your operation or not, not everybody has that, you can still adopt a scientific process to study the failures that you have. Mark and I are often surprised that some outside hydraulic shop dropped in, told them what they found broken and left with the broken parts and that nobody at that shop or that plant got to see the supposedly failed parts and got to participate in any kind of lasting learning. That’s a very unfortunate place to end up.

Mark P.: Oh yeah, because we would often see a valve saying well the valve was strapped, so we gave you a new one. Well, let me see the parts. My father grew up in the automotive industry back in Detroit, and I remember the change, where you had to keep parts so a customer could see them. Well, we need to do that industrially as well. Have a look at the parts. See if you can analyze what fails. We had a situation where we had a piston pump, multiple piston pump failures on a piece of oilfield equipment, and when we took the last pump apart, we still showed all nine pistons used selected into the rotating group, but we found an extra shoe in the case, where it had came back through a case drain line, or suction line. Somehow, there was a complete bronze piston shoe left in the case of a pump. So when we’re getting to that root cause, we never understood how they thought they could clean that system out in the field. But after the third pump failure, somebody found time to bring it in and take a closer look at it.

Carl Dyke: Again, staying with the process of looking for the simplest and most basic causes first. Very easy to be pulled into too much complexity. Yeah, send photographs. It’s very easy with our mobile phones these days and cameras out at the site, to take a photograph of a failed component, something that doesn’t look quite right, some wear that occurred internally. You may have to put the system back together, but at least you could take a quick shot of that and send that photograph off to perhaps the manufacturer of your machine and ask them to help you analyze it. If they’ve sold you expensive equipment, or it could be the component maker themselves, pull them into the process and say “Hey, help me with these failures I’m having. What could be the root cause? I wanna make this type of failure go away.”

Mark P.: And as we showed there, contamination … I can look back at my very first hydraulic manual way back in those days, and I still see it today in manuals produced by some of the major manufacturers, training manuals, and they’ll say at the beginning, 70, 80% of the hydraulic failures are contamination related. And then they go on through the rest of the book to talk about the components, but nobody’s really jumped in there and told me how to control contamination. But in the age that we have today of much more sophisticated valves, I mean, when we’re back into gear pumps and valves that we pull by hand, we could overcome a lot of contamination issues by just pulling harder. Today, with servo control valves, with proportional [inaudible 00:41:58] that don’t have the strength, the least little bit of contamination is going to stick and stop a valve from functioning properly.

Carl Dyke: So with the machine back together, with the repair made, when operations calms down that they have their productivity back, this is a great opportunity for maintenance, and the management people from the operations side to get together and continue their root cause analysis and work backwards. Here’s one example. This is an extremely simple example, but it’s an example of how the simplest things sometimes trip us up and cause us to make business decisions that forever then impact the reliability of a hydraulic system. We had a client who had a pump that they felt had seized, it had failed, and that the electric motor could no longer turn that hydraulic pump. It had thrown the electric motor protection in the plant. The breakers off. And of course they observed that there was a coupling failure as well, and the assumption was that the pump had seized, that it couldn’t be turned. And they sent that pump out for repair, and it was rebuilt and sent back and failed again, several weeks later.

Well, a series of why questions needs to be asked at this stage. Was the pump not repaired correctly? Well, they were unsure. They couldn’t be verified. They simply got an invoice from the hydraulic shop that the pump had been repaired. It was a single line invoice, no real notes as to what process was followed. Alright, so how was it determined that the pump was seized? Well, going back to the plant technicians, they said “Well, the flexible inserts in the pump-to-electric-motor coupling had failed.” The electric motor couldn’t turn the pump. Well the question was asked “How does this prove the pump failure?” Well the group was able to ascertain that that clearly does not prove a pump failure. It only proves that the coupling itself failed.

Mark P.: Sure if you had a seized piston pump, you have the thought that the relationship that you have with the repair shop should be telling you “Whoa, what happened here? We found six of the pistons seized in the board and nothing could turn in the shaft.” And that could explain a bad drive coupling. But a broken or a bad drive coupling doesn’t prove that a pump is seized.

Carl Dyke: So in this case, yes, the question was how was the decision made then to repair the pump? Well, the maintenance technicians in the plant simply couldn’t think of any other cause. The question was asked “Did anyone examine coupling failure as a cause in detail?” No. Never occurred to them to look at that coupling as the source of the failure. When they went to the parts list for the coupling for that machine and checked its model and the rating of the stiffness of the insert, the flexible composite materials, it was found that the coupler insert that was apparently being used did not match with bill of materials for the original. Why was that? Well, turns out that the original was not available at one point when a maintenance routine required that that coupling insert simply be replaced. So they went with a different one. And company stores updated the list to their standard parts to use the new softer insert material. Why was that? The maintenance department had informed the company’s part stores that the new coupling insert works just fine. You can see how simple this is.

Mark P.: And you also see how subjective some of these answers were here. They were not objectively shown. Somebody didn’t show a data sheet that said a 50 horsepower motor drive in this pump requires X coupler and insert.

Carl Dyke: It had been on the original specs, but yeah, they lost track of that information.

Mark P.: And we’ve seen that in filters all the time. People will replace the filter without ever checking its actual data rating or something to that effect, and say “Well, this one fits, so this one’s cheaper. We might as well use this one.” And six months down the road when you have contamination issues, you realize how that really was the root cause.
Carl Dyke: So this was a really simple example. Really, what you could see here that in this case, it was more of a human failure and a business system failure more than it was any kind of a physical cause. But these are the types of simple things that can trip us up if we’re not carefully keeping track of everything, even those early things that Mark and I talked about, like the documentation and the correct components and parts. So it takes a fair amount of education of maintenance persons. In this case, that’s what the client had to do, educate their maintenance crew as to why the original component specifications can be absolutely critical.

Mark P.: Yeah, we’ve actually, working on this lab press project right now, and here’s a machine that has been running for 28 years, two of them side by side and I was asked to find out what’s different between them and I find out that once I acquired, through much effort, some original documentation, I noticed that neither press had the same directional valve that was required by the factory in the first place. And so why is one press have one type of control and one has the other? And who made that decision? But for right now, we’re just taking time to observe what are the differences between here? The questions are still being asked. Who knew how that got changed? Well, over time, maintenance personnel changed. Nobody knows the history. And at some point, you’re either starting from square one today and somebody has to take the time to find out if there’s a right part.

Carl Dyke: Absolutely, Mark. Time spent in the careful, methodical, investigation absolutely pays off in correcting faults and failures.