Not Unexpected

The Benefit Of Prior Investigative Experience And A Trained Eye

Preface

A large worldwide financial institution plans the installation of two 4 in. wet taps into their active 24 in. condenser water header piping which serves their refrigeration machines and heat exchangers.  As a standard precaution, they hire CorrView International, LLC to determine the wall thickness of the pipe in the areas planned for welding.  CorrView has previously inspected this same facility going back 18 years, and has consistently identified low corrosion activity at their condenser water system to below 1 MPY.

Investigation

Testing in both well defined areas takes under ½ hour to perform and  produces excellent results as expected.  We confirm high and uniform wall thickness still exceeding new standard pipe specifications of 0.375 in. since the pipe had been originally oversized when installed 22 years earlier.  In combination with low corrosion activity, virtually unlimited service life remains in both areas of concern.  Prior testing of the 24 in. mains had consistently identified high wall thickness.  Our initial measurements of the piping system 18 years earlier had identified high wall thickness values to near 0.400 in. – to define that oversized standard grade pipe had been installed.

During our visit we notice that the facility is running approximately 4,000 tons of refrigeration while outdoor temperatures are near 8° F.  Two large plate and frame heat exchangers are available but not in service.  When we inquire why the heat exchangers are not in use on such a cold day we are informed they have never ever been used throughout the entire history of the facility.

A second look at the piping layout shows that the 12 in. distribution lines to and from the heat exchangers run an approximate 50 ft. distance across the mechanical room to the opposite wall.  Supply and return lines to the heat exchangers are the 1st and 2nd take-offs to the flow of supply water from the cooling tower, and are bottom connections to the 24 in. main lines.  Isolation valves are not installed at the mains, but at the heat exchangers approximately 50 ft. away.

Recognizing this piping arrangement as a common problem area and as a significant potential threat, we spot test areas of all four piping runs and quickly identify very severe pitting along the bottom and lower sides.  Ultrasonic testing at the top of all four lines produces measurements at near 0.350 in. to 0.365 in., and very near new ASTM standard pipe specifications of 0.375 in.  Alternately, inspection of the bottom of the pipe produces much lower wall thickness readings averaging near 0.125 in., with lowest thickness values to near 0.090 in.  Low thickness values are marked onto the pipe itself should the facility have the interest to confirm our results.

Considering the possibility that there may be some error in the instrumentation, we recalibrate the probe, changes probes and recalibrate again, and then switch to another instrument where we produce the exact same readings.  Low wall thickness is localized beginning at the bottom of the 90° elbow from the main to approximately 6 ft. outward from the elbow.  Higher than normal corrosion activity is identified further out to the heat exchangers, but with wall thickness values still near 0.300 in., is of no consequence .

Our testing identifies the highest deterioration at the two supply side take-offs, with slightly higher but still unacceptable wall thickness at the return side piping.  Three hours of further investigation is provided to the facility, and to a critical services long term client of ours at no charge.

The engineering staff is requested over to the site of our findings, where further testing and demonstration is performed in order to give them a first hand understanding of their conditions.  We explain that the severe bottom pitting is likely due to over 20 years of rust and other airborne particulates settling in this area of pipe.  Installing isolation valves far downstream at the heat exchangers, instead of immediately at the supply and return headers themselves as more appropriate, has created a large no flow settling zone.  In addition, the installing the elbows from the mains at the bottom of the 24 in. header, instead of at the side wall or at top, guaranteed that any material traveling down the 24 in. line from the towers would fall down into the 12 in. outlet.  Both are serious design errors in our view, but common ones seen at most larger data center and larger chiller plant operations.

The facility is a 24 /7 computer data storage site which is designed for uninterrupted service and which cannot be shut down.  However, it is very clear that such low wall thickness to the bottom of the pipe, and a continued acceleration of that loss, will prematurely cause its shutdown, and at some unexpected moment.

The decision to shut down the facility is made, and requires almost 9 months to coordinate.  A 6 hour window is provided to remove all 4 elbows and weld in their place 12 in. weldolets to isolation valves.

Once offline and drained, welders find they cannot burn or cut the pipe at the elbow.  Two acetylene torches trained onto the same area of pipe fails to produce any result.  A member of the mechanical firm climbs to the top of the pipe and turns a hole to reveal that the pipe is filled to approximately 70% of its depth with rust and mud.  Obviously, almost 8 in. of thick wet mud and rust is absorbing the heat applied to the bottom of the pipe.

Attempts to shop vac the rust out from the top  hole is unsuccessful since the material is packed rock solid into the pipe.  A hammer drill is then employed to break up the blockage, after which it can be removed by vacuum.  With the rust removed, the pipe can be cut away.

This unexpected problem causes panic to everyone given that an absolute maximum 6 hour window has been provided to complete operations on this and 3 other 12 in. lines and have the entire refrigeration system back up and running, and that the clock is ticking down.  Additional teams of  mechanics are rushed in from other Sunday projects to assist.

All four lines are found to be in similar condition.  A later inspection of the removed pipe shows it extremely thin as had been reported by our investigation.  New 12 in. valves are installed right at the headers themselves and the system returned to service within the 6 hour window allowed.

Conclusion

Facility management is so astounded at the event that they provide CorrView International, LLC a purchase order for a full week of on-site testing in order to inspect any other piping to our interest.  They express that such a finding raises fears to other potential and unknown threats.

Our further efforts identify some significant threats at both the fire sprinkler and condenser water systems, which are addressed next by facility management thereby avoiding unnecessary failure.