Signalling shutdown - Croydon area - Wednesday night 18.12.19

[whole diary]

[grahame]

Fascinating insight from Netwok Rail into a shutdown in the Croydon area on Wednesday.

Disruption at Victoria and London Bridge:
how it happened and what we are doing about it

John Halsall, Managing Director, Southern region

On Wednesday night, the signalling system that controls the safe movement of trains through Croydon shut itself down to protect itself from an external power supply problem.

A power surge from our external provider put too many volts into our supply for around 20 seconds.

Our system is designed to fail-safe, so every signal – the traffic lights of the railway – went red and every train was brought to a stand.

The result was that for almost an hour we were unable to run services on the busiest section of the UK^▸ (United Kingdom)’s railway, causing disruption that lasted until Thursday morning.

[continues]

[broadgage]

Was there any effect on electrical equipment outside of the railway ? Or was this another case of railway equipment being highly sensitive to minor electricity supply fluctuations that no one else noticed.

For signalling equipment of such importance, was no standby supply available? At the very least, I would expect a UPS with a battery run time of some minutes, and with a duplicated mains voltage supply into the UPS.

[SandTEngineer]

Was there any effect on electrical equipment outside of the railway ? Or was this another case of railway equipment being highly sensitive to minor electricity supply fluctuations that no one else noticed.

For signalling equipment of such importance, was no standby supply available? At the very least, I would expect a UPS with a battery run time of some minutes, and with a duplicated mains voltage supply into the UPS.

If you read it again it says it was a power surge, not a loss of supply.  Electronic signalling equipment has strict set voltage limits and if these are exceeded the electronic equipment involved will self isolate to protect it from all being blown to pieces......   Normally things will reset after the supply restores to normal limits, but perhaps in this instance it didn't for some time due to other external factors.

[onthecushions]

If the CB opens then there is certainly a loss of supply, however caused, hi-volts or lo-volts.

If there was uninteruptable power available to turn the signals red then there was the same power around to maintain the system.

I despair of electrical engineers.

OTC

[SandTEngineer]

If the CB opens then there is certainly a loss of supply, however caused, hi-volts or lo-volts.

If there was uninteruptable power available to turn the signals red then there was the same power around to maintain the system.

I despair of electrical engineers.

OTC

Lineside signalling equipment is not usually supported by a UPS, but by a standby diesel generator.  We don't generally utilise Circuit Breakers but use hard wired fuses which are not susceptable to voltage/frequency flucuations, but only to excessive currents.

Does your despair also apply to Signal Engineers......

[SandTEngineer]

I don't do TWITTER but apparently an explanation of what happened has been posted by NR^» (Network Rail - home page) here which seems to be the same info a in the link posted by Grahame in the opening topic post: https://mobile.twitter.com/networkrailse/status/1207703163358392322?s=21

[IndustryInsider]

Very impressive explanation and apology from NR^» (Network Rail - home page) there.

[ellendune]

Very impressive explanation and apology from NR^» (Network Rail - home page) there.

For those who don't do twitter it includes a link to a longer explanation on their website

[onthecushions]

[

Does your despair also apply to Signal Engineers......

No. I think they/you are marvellous, making me and mine blissfully safe in rail travel. Many thanks.

A pity S&T^▸ (Signalling and Telegraph) types don't train highways designers.

Season's greetings,

OTC

[broadgage]

Was there any effect on electrical equipment outside of the railway ? Or was this another case of railway equipment being highly sensitive to minor electricity supply fluctuations that no one else noticed.

For signalling equipment of such importance, was no standby supply available? At the very least, I would expect a UPS with a battery run time of some minutes, and with a duplicated mains voltage supply into the UPS.

If you read it again it says it was a power surge, not a loss of supply.  Electronic signalling equipment has strict set voltage limits and if these are exceeded the electronic equipment involved will self isolate to protect it from all being blown to pieces......   Normally things will reset after the supply restores to normal limits, but perhaps in this instance it didn't for some time due to other external factors.

I DID^» (Didcot Parkway - next trains) read it properly. A UPS protects not only against complete loss of power, but also against voltage out of tolerance as was reported to be the case.
If the mains supply is present but with an excessively high or low voltage, then a UPS will either correct this via a built in transformer, or will revert to battery power.

[Surrey 455]

I was sitting on a very delayed train at London Bridge that night which was cancelled and later re-instated (although about an hour later) after I had got off and gone to Waterloo instead . I was impressed with the scrolling information screens on that train, a Southern 377. This told me that all trains would be delayed or cancelled due to power supply failure in the East Croydon area. A few days earlier I was on another Southern 377 when approaching Epsom, the scrolling display mentioned that there were no SWR^» (South Western Railway - about)  trains between Epsom and Ewell West because of a landslip.
I have never seen this up to date information on any other train before. Is it keyed in manually or transmitted from a control centre?

[stuving]

I was sitting on a very delayed train at London Bridge that night which was cancelled and later re-instated (although about an hour later) after I had got off and gone to Waterloo instead . I was impressed with the scrolling information screens on that train, a Southern 377. This told me that all trains would be delayed or cancelled due to power supply failure in the East Croydon area. A few days earlier I was on another Southern 377 when approaching Epsom, the scrolling display mentioned that there were no SWR^» (South Western Railway - about)  trains between Epsom and Ewell West because of a landslip.
I have never seen this up to date information on any other train before. Is it keyed in manually or transmitted from a control centre?

It will be sent by radio, but perhaps set up in advance and triggered.

Bombardier's in-house CIS^▸ (Customer Information System) makers BTROS offer a Real Time Passenger Information System (RTPIS), of which they say:

Unlike the current PIS^▸ (Passenger Information System) Systems, where the on-train modems communicate directly with a single remote ground based system fitted with a dial-up modem, the RTPIS converted trains communicate via an intermediate secure hosted Transmission Router. This arrangement enables RTPIS uploads, RTPIS downloads, route database uploads and GPS tracking operations to be available concurrently from multiple ground based support locations, enabling quicker and easier updates to the PIS system.

However, from 2013 if not earlier, the OTMR^▸ (On Train Monitoring Recorder) (on-board train monitoring recorder) on 377s had developed a bit and might do that already: "The OTMR has ship to shore capability and a real time passenger information system. GPS is used to verify the train’s position so that relevant messages can be transmitted for the location. This is being applied system-wide." (Piers Connor for Modern Railways March 2013)

But if it's Bombardier standard kit, almost new 387s could have it too.

[Electric train]

I don't know the specifics of this particular event, but I do know something about the power supplies on the ***Southern Region*** of NR^» (Network Rail - home page)

First the term power surge is used in the rail industry by folks in the operating department, a true power surge would fry a lot of equipment other than just the signalling. It was more likely to have been a short loss of power, just a few milliseconds is sufficient for the signalling system to go into a fail safe mode.  Basically, many of the modules used in signalling have "crowbar" protection, this device shorts out the supply into the module if it sees voltage transient's or an unstable supply and blows the fuse in the module; the signalling will fail safe but this will stop or restrict the movement of trains until a tech has attended site.

NR ***Southern Region*** takes its power from a number of DNO^▸ (Distribution Network Operator) and TNO^▸ (Transmission Network Operator) Grid sites a fault on their system of on NR's own internal 33kV distribution network can be sufficient to cause the above yet to any user they might have notice the light dip a bit.

There are UPS around the NR signalling supply system but these are not always installed, a complex set of sums used by commercial people of ten overrides the Engineers choice.


*** as an Western Region (BR^▸ (British Rail(ways))) it uncomfortable working in the Southern Region 

[ellendune]

First the term power surge is used in the rail industry by folks in the operating department, a true power surge would fry a lot of equipment other than just the signalling. It was more likely to have been a short loss of power, just a few milliseconds is sufficient for the signalling system to go into a fail safe mode. 

Why would NR^» (Network Rail - home page) say it was an over-voltage fault lasting 20 seconds or more then?

[onthecushions]

There are UPS around the NR^» (Network Rail - home page) signalling supply system but these are not always installed, a complex set of sums used by commercial people of ten overrides the Engineers choice.

I'm intrigued but not surprised to learn the actual age of the railway's commercial people.

Greetings!

OTC