Train Graphic
Great Western Passengers' Forum Great Western Coffee Shop - [home] and [about]
Read about the forum [here].
Register and contribute [here] - it's free.
article index - [here]
 today - MRUG meeting
today - ACoRP board nominations close
18/10/2019 - TravelWatch SouthWest
18/10/2019 - GWR meet the team - Westbury
19/10/2019 - MRUG meet and chat
21/10/2019 - Ticket booking test - BRI
Random Image
Train Running @GWR Twitter Acronyms/Abbreviations Station Comparator Rail news GWR co. site Site Style 1 2 3 4 Chat on off
Next departures • Bristol Temple MeadsBath SpaChippenhamSwindonDidcot ParkwayReadingLondon PaddingtonMelksham
Exeter St DavidsTauntonWestburyTrowbridgeBristol ParkwayCardiff CentralOxfordCheltenham SpaBirmingham New Street
October 16, 2019, 12:32:58 pm *
Welcome, Guest. Please login or register.

Login with username, password and session length
Forgotten your username or password? - get a reminder
Most liked recent subjects
[217] Weather disruption caused in 2014, and how to prevent it happe...
[77] Rail network enhancements pipeline
[56] Railway bridges struck by road vehicles - merged topic, ongoin...
[53] On crossing borders by public transport - to and within the UK...
[49] Major Signalling Issue affecting services between Brockenhurst...
[46] Helping (?) people in wheelchairs
News: A forum for passengers ... with input from rail professionals welcomed too
 
   Home   Help Search Calendar Login Register  
Pages: 1 2 [3] 4
  Print  
Author Topic: Power outage strands trains  (Read 3696 times)
broadgage
Transport Scholar
Hero Member
******
Posts: 2811



View Profile
« Reply #30 on: August 20, 2019, 03:54:46 pm »

The report does IMHO make very damming reading for the railway industry.
It states that National Grid did not interrupt ANY traction current supplies. 25Kv AC traction current remained live throughout.
A couple of 750 volt DC traction current supplies failed, but only due to the RAILWAY OWNED transformer/rectifier units tripping out when the grid frequency was below normal.

The railway chaos as widely reported therefore seems to have two principle causes.
Firstly the failure of the new Thameslink EMUs to function with a lower than normal line frequency. It seems that this was NOT a "failure to re-start after the power cut" as was initially reported, there was no failure of the power supply. The new trains simply stopped working when the frequency dropped and could not be re-started by the driver when the frequency returned to normal.
This to me looks like a grossly defective design if it effectively closes a main line for hours after a brief and entirely foreseeable drop in grid frequency.

Secondly, the failure of power supplies to signalling controls and to stations. These really should have standby supplies that actually work when called upon and permit of continuation of normal train services.
The larger and more important facilities ought IMHO to have TWO standby supplies, one from the traction current and one from an "electrically distant" part of the national grid, in addition to the normal supply.
Smaller and lower priority should at least have one main supply and one standby supply.
In non electrified areas, a diesel generator is an alternative to the traction current.
Logged

A proper intercity train has a minimum of 8 coaches, gangwayed throughout, with first at one end, and a full sized buffet car between first and standard.
It has space for cycles, surfboards,luggage etc.
A 5 car DMU is not a proper inter-city train. The 5+5 and 9 car DMUs are almost as bad.
SandTEngineer
Transport Scholar
Hero Member
******
Posts: 3007


Climbing a Pole Somewhere in the far Southwest


View Profile
« Reply #31 on: August 20, 2019, 04:20:41 pm »

The report does IMHO make very damming reading for the railway industry.
It states that National Grid did not interrupt ANY traction current supplies. 25Kv AC traction current remained live throughout.
A couple of 750 volt DC traction current supplies failed, but only due to the RAILWAY OWNED transformer/rectifier units tripping out when the grid frequency was below normal.

The railway chaos as widely reported therefore seems to have two principle causes.
Firstly the failure of the new Thameslink EMUs to function with a lower than normal line frequency. It seems that this was NOT a "failure to re-start after the power cut" as was initially reported, there was no failure of the power supply. The new trains simply stopped working when the frequency dropped and could not be re-started by the driver when the frequency returned to normal.
This to me looks like a grossly defective design if it effectively closes a main line for hours after a brief and entirely foreseeable drop in grid frequency.

Secondly, the failure of power supplies to signalling controls and to stations. These really should have standby supplies that actually work when called upon and permit of continuation of normal train services.
The larger and more important facilities ought IMHO to have TWO standby supplies, one from the traction current and one from an "electrically distant" part of the national grid, in addition to the normal supply.
Smaller and lower priority should at least have one main supply and one standby supply.
In non electrified areas, a diesel generator is an alternative to the traction current.


You obviously didn't read the previous posts by Electric Train and myself........ Roll Eyes
Logged

Out of this nettle, Danger, we pluck this flower, Safety.
[Henry IV, Part 1, Act 2, Scene 3]
SandTEngineer
Transport Scholar
Hero Member
******
Posts: 3007


Climbing a Pole Somewhere in the far Southwest


View Profile
« Reply #32 on: August 20, 2019, 04:27:26 pm »

From the ESO Report: https://www.nationalgrideso.com/sites/eso/files/styles/content_embedded_image/public/images/Power%20cut%20timeline%20infographic%20jpeg.jpg?itok=0j8Jyt23



....and the Rail specifics:

Quote
Network Rail and the DNOs have confirmed that no track supplies were lost due to the DNO’s LFDD protection operation. However, there were significant impacts on the rail network during the event as noted below:

1. While the built-in resilience of Network Rail’s electrical power infrastructure meant traction power was maintained to the vast majority of the railway throughout the incident, there were frequency-related trips which occurred at two DC traction locations which Network Rail are investigating.

2. Eight signal power supplies in principally rural locations suffered minor outages with minimal passenger impact. Network Rail are reviewing resilience at these locations.

3. Class 700 and 717 trains shut down north of Farringdon and Kings Cross stations due to their internal protection systems being triggered. The Network Rail overhead line power supply operated continually. The shutdown of these trains had a knock-on impact by delaying all other trains behind them requiring the temporarily closure of London St Pancras and Kings Cross stations which led to Friday rush hour overcrowding.

Govia Thameslink Railway (GTR) have advised that their Class 700 and Class 717 trains that were operating on AC power were affected by the frequency deviation below 49Hz. They are investigating with the manufacturer the reason for the trains stopping when the frequency dropped below 49Hz. Of the approximately 60 Class 700 and Class 717 trains operating on overhead power supply, half were restarted by the driver and the remainder required a technician for reset. All other GTR classes of train were unaffected. GTR will provide further information, relevant to the technical report, once their investigation has concluded.

The impact to the rail network was that thousands of passengers had their journeys delayed with 371 trains cancelled, 220 part cancelled, and 873 trains delayed. London St Pancras and King’s Cross stations had to close for several hours due to overcrowding and London Euston went exit only for a period of time.

London Underground have confirmed there were impacts on the London Underground Victoria Line, which was
suspended as a result of the event and service was restored at 17.35.
« Last Edit: August 20, 2019, 04:39:49 pm by SandTEngineer » Logged

Out of this nettle, Danger, we pluck this flower, Safety.
[Henry IV, Part 1, Act 2, Scene 3]
grahame
Administrator
Hero Member
*****
Posts: 26737



View Profile WWW Email
« Reply #33 on: August 20, 2019, 05:07:04 pm »

From the Daily Mail

Quote
Lightning strike DID cause mass power cuts before new German-built trains repeatedly shut down bringing chaos to the railways, official report reveals

Some 60 German-built trains caused carnage on the railways during a huge power cut by repeatedly shutting down despite the overhead wires lines working.

Thousands of rail passengers had their journeys delayed after major issues with the Class 700 and 717 Siemens trains operated by Thameslink in South East England.

The stock, built in Krefeld near Düsseldorf, 'reacted unexpectedly to the electrical disturbance' - and half of them required an engineer to be sent out to reset the train.

Does the country of manufacture really have a big bearing on why the trains shut down?
Logged

Coffee Shop Admin, Vice Chair of Melksham Rail User Group, on the board of TravelWatch SouthWest and of RailFuture
patch38
Transport Scholar
Hero Member
******
Posts: 578


View Profile
« Reply #34 on: August 20, 2019, 06:01:32 pm »

The Germans built what was specified. The Daily Wail will need to hunt down the nationality of the designer if they want to find someone to vilify.
Logged
stuving
Transport Scholar
Hero Member
******
Posts: 4175


View Profile
« Reply #35 on: August 20, 2019, 06:08:27 pm »

I think Broadgage overstates the case for the prosecution (with one obvious exception).

One thing which I think is clear in the interim report, though not stated as such (it's not a text book), is that interaction between connected machinery at all levels matters. In a sense, the network is only the sum of all its parts, large and small. Some parts NG can control or at least influence, but some have to do the right thing automatically, and must be designed for that.

So big generators must stay connected as the frequency falls, or even increase their output. That's what stabilises the system. And they mustn't disconnect when they see a transient from routine network operation, like that trip/reclose caused by lightning. But designing high-power systems to meet that is difficult and expensive, in part because if you get it wrong your machine will disassemble itself and disappear in all directions (or, if electronic, vaporise - which is true at much lower power levels too).

Outside the grid itself, most loads and sources won't be designed to such exacting standards, so will err on the safe side - hence the loss of so much 'embedded generation'. I think the supply industry's own rules even dictate it, since they can't trust such 'domestic' devices to be designed to their own higher standards. These sources normally feed local loads, which looks to the grid like a reduced load in the distribution network there. Presumably it would help if more local loads disconnected to balance that effect - that 500MW lost was as much as a big generator.

What's left in between is critical loads, where losing power has consequential effects that worry people, and aggregate effects. Those 700/717 trains are a bit of both. Somewhere in the design/requirements analysis/specification process, when looking at the section headed "response to supply transients", there is an important question to be asked: "what if a lot of trains throughout the network behave like this all at once? Is the expected response acceptable?". It's an obvious and necessary question for a railway, simply because it is a network - and a Chinese puzzle to be solved continuously. So that's a fail.

For railway power and signals, it depends on the effect and the time taken to restore operation. I don't see that demanding full uninterrupted operation through any transient event or supply loss is necessary. In part, that's because it's not going to happen - somewhere, something will trip - so concentrate in minimising the scope and duration of interruptions.

But for other systems, and even for Ipswich Hospital (for which I have little information), turning off (some loads, at least) and not reconnecting for a minute of two may be fine. It is, after all, a rare event. Just provided the back-up systems work (and as we know...), and reconnecting doesn't call for an hour-long communal manual-reading session.
Logged
stuving
Transport Scholar
Hero Member
******
Posts: 4175


View Profile
« Reply #36 on: August 20, 2019, 06:14:55 pm »

And to answer my own question about battery storage... I was expecting some, but not as much as 472 MW. Apparently it was 314MW at the start of the year, and additions in planning or construction will/would take that to over 5 GW. So it's obviously really fashionable just now. A lot of it is buffering wind power and other renewables, but some is just being built to offer NG fast response to help in exactly this kind of event (i.e. 9/8/19).

And NG did a study during 2014/15 into trialling this for use in their "Enhanced Frequency Control Capability", with as objectives:

Quote
The Main Drivers for using battery storage
  • Demonstrate the principle [of?] operability of a frequency control battery on the network.
  • Demonstrate emulation of rotating generators and their inertia by implementing a very high response rate (milliseconds or tens of milliseconds).
  • Demonstrate different reaction speeds.

That 'emulation' they also label "virtual inertia". I gather that implementing the control of power and phase based on frequency (in a DC-to-3 phase converter) to do this safely and reliably is still work in progress.
Logged
broadgage
Transport Scholar
Hero Member
******
Posts: 2811



View Profile
« Reply #37 on: August 21, 2019, 05:25:21 pm »

Re post #31, yes I did read previous posts on this subject.

Looking at the scale of the chaos I still maintain that backup power supplies for signalling were insufficient. Lineside signals remote from control rooms also need back up power supplies.
Up to a few miles distant, they could use power from the control center. Distribution of signalling power at 650 or 690 volts used to be the norm.
When this is not viable then battery backup should be considered.

I also stand by my remarks that the new Thameslink trains are of a defective design if they can not tolerate a relatively rare but entirely foreseeable drop in line frequency.
Is this design feature to be included in the Crossrail trains ? The thought of thousands of passengers trapped below ground for hours after a brief drop in frequency is rather worrying.
Logged

A proper intercity train has a minimum of 8 coaches, gangwayed throughout, with first at one end, and a full sized buffet car between first and standard.
It has space for cycles, surfboards,luggage etc.
A 5 car DMU is not a proper inter-city train. The 5+5 and 9 car DMUs are almost as bad.
IndustryInsider
Hero Member
*****
Posts: 7569


View Profile
« Reply #38 on: August 21, 2019, 06:39:53 pm »

Different manufacturer (Bombadier not Siemens), so likely to behave differently?  The Siemens units could receive tolerance modifications as well surely?

Emergency evacuation of Crossrail trains in tunnels should be a relatively simple matter compared with traditional underground trains.
Logged

To view my GWML Electrification cab video 'before and after' video comparison, as well as other videos of the new layout at Reading and 'before and after' comparisons of the Cotswold Line Redoubling scheme, see: http://www.dailymotion.com/user/IndustryInsider/
stuving
Transport Scholar
Hero Member
******
Posts: 4175


View Profile
« Reply #39 on: August 21, 2019, 07:18:18 pm »

As I said, I don't see disconnection of trains, or some other railway equipment, as unacceptable given the very fast frequency drop to below 49 Hz. That's provided these things are all back working within a minute or two, which as far as I know was the case for NR's signalling (at least on bigger lines) and DC traction.

Those 700/717 trains, however, didn't - and that is inexcusable. If it is "only software", that only makes it worse, and I'm sure it was a known fault, in the sense that more than one driver had been unable to reset and restart a train. And while trainmakers might source some bits of software from suppliers of their competitiors too, in general each will have its own herd of bugs. No doubt we'll find out what the 345s' ones are.

I find it a bit odd that even some well-qualified commentators seem to think that because this event was more extreme than 20 years ago due to the change in generation on the grid, that means NG don't realise that's happening and have no plans to cope with it. You could infer they are a bit late in their implmentation, perhaps. But even if that figure I found of 4.9 GW of battery capacity in the pipeline is largely wishful planning, and say 1 GW is added by the end of 2020, NG will be able to cope with an event like this one using batteries alone.

NG are in a way being criticised for not explaining what their evolution plans are, in a report that wasn't meant to be about that. But one thing I think ought to have been explained is why the recovery of supplies took half an hour after the grid was stabilised. Was that just the way the DNOs do things, in which case could it be done quicker? Or was the rate of demand rise dictated by NG  to match the rate of rise of available generation?
Logged
Electric train
Hero Member
*****
Posts: 3116


The future is 25,000 Volts a.c.


View Profile
« Reply #40 on: August 21, 2019, 08:10:38 pm »

The report does IMHO make very damming reading for the railway industry.
It states that National Grid did not interrupt ANY traction current supplies. 25Kv AC traction current remained live throughout.
A couple of 750 volt DC traction current supplies failed, but only due to the RAILWAY OWNED transformer/rectifier units tripping out when the grid frequency was below normal.

The railway chaos as widely reported therefore seems to have two principle causes.
Firstly the failure of the new Thameslink EMUs to function with a lower than normal line frequency. It seems that this was NOT a "failure to re-start after the power cut" as was initially reported, there was no failure of the power supply. The new trains simply stopped working when the frequency dropped and could not be re-started by the driver when the frequency returned to normal.
This to me looks like a grossly defective design if it effectively closes a main line for hours after a brief and entirely foreseeable drop in grid frequency.

Secondly, the failure of power supplies to signalling controls and to stations. These really should have standby supplies that actually work when called upon and permit of continuation of normal train services.
The larger and more important facilities ought IMHO to have TWO standby supplies, one from the traction current and one from an "electrically distant" part of the national grid, in addition to the normal supply.
Smaller and lower priority should at least have one main supply and one standby supply.
In non electrified areas, a diesel generator is an alternative to the traction current.


As I said, I don't see disconnection of trains, or some other railway equipment, as unacceptable given the very fast frequency drop to below 49 Hz. That's provided these things are all back working within a minute or two, which as far as I know was the case for NR's signalling (at least on bigger lines) and DC traction.

Those 700/717 trains, however, didn't - and that is inexcusable. If it is "only software", that only makes it worse, and I'm sure it was a known fault, in the sense that more than one driver had been unable to reset and restart a train. And while trainmakers might source some bits of software from suppliers of their competitiors too, in general each will have its own herd of bugs. No doubt we'll find out what the 345s' ones are.

I find it a bit odd that even some well-qualified commentators seem to think that because this event was more extreme than 20 years ago due to the change in generation on the grid, that means NG don't realise that's happening and have no plans to cope with it. You could infer they are a bit late in their implmentation, perhaps. But even if that figure I found of 4.9 GW of battery capacity in the pipeline is largely wishful planning, and say 1 GW is added by the end of 2020, NG will be able to cope with an event like this one using batteries alone.

NG are in a way being criticised for not explaining what their evolution plans are, in a report that wasn't meant to be about that. But one thing I think ought to have been explained is why the recovery of supplies took half an hour after the grid was stabilised. Was that just the way the DNOs do things, in which case could it be done quicker? Or was the rate of demand rise dictated by NG  to match the rate of rise of available generation?

The most likely cause for signalling, CIS, even some rectifiers etc tripping out in this event was not frequency instability but the dip in voltage on the National 400kV system, it is an "infinite busbar" hence a short circuit almost anywhere in the country will be reflected Nationally.   Most systems will either not see the dip or will self reset.  At major stations only essential systems related to safe evacuation of the station are backed up by a UPS or generator; to install the generator capacity to power even just keep Paddington normally operational (ie without retail) would require a 1 MW generator.

Regards the class 700's allegedly there was a problem with Siemens own computer and communications systems at the depot suffering from the dip in the power supplies which I a lead to believe prevented the reset to be sent via GSM-R.

Like all these events there are work streams across the railway industry (NR and TOC's) looking at the lessons learnt and what can be done should something like this happen againg
Logged

Mark Carne 26 June 2015 - "The challenges of delivering myriad improvement projects while still running a railway seven days a week were simply overwhelming".
Oxonhutch
Transport Scholar
Hero Member
******
Posts: 659



View Profile
« Reply #41 on: September 10, 2019, 03:07:00 pm »

Technical Report of the National Grid ESO has been released.

https://www.ofgem.gov.uk/system/files/docs/2019/09/eso_technical_report_-_final.pdf
Logged
stuving
Transport Scholar
Hero Member
******
Posts: 4175


View Profile
« Reply #42 on: September 10, 2019, 07:07:38 pm »

The appendices referred to in the main report are also available from Ofgem here.

Of the three major failures that contributed to this event, the root causes were:

Hornsea wind farm - instability in the system that controls power (real and reactive) flows, leading to oscillatory power flows and excessive currents that in turn led to a shutdown. That's software, which Orsted say has been upgraded.

Little Barford CCGT generator - the first shutdown, of the steam turbine, is as yet unexplained but could well be software (RWE describe it as "a discrepancy in the speed signals"). The shutdown of the two gas turbine was presumably inevitable in the long run, but was premature due to excessive steam pressure when the turbine was bypassed (and stem sent direct to the condenser). Unlikely to be software, unless it is very large, heatproof, and nesting in the steam pipes.

Govia's class 700 and 717 trains - for the 30 or so that needed a technician+laptop to get them going, undoubtedly software. Indeed, it was an "upgrade" that removed a feature necessary to meet the specification that made these ones unable to do what the other 17 did, being rebooted by the driver. So not just software, but an own goal. Siemens are going to patch that, but are unwilling to simply revert to the previous version, because that was needed to remove some other undesirable behaviours.

I do wonder whether it is really necessary for any external fault (not implying the input converter is itself faulty) to cause a lockout, or for its reset to take 10 minutes. But there you go, that's progress, isn't it?

So it could be 3/3 down to duff software. Welcome to the 21st century!

I'm also a bit puzzled that NG say the load-shedding worked as planned, when the disconnection of 931 MW of load was offset by so much local generation being lost (disconnected by the DNO or itself) that the nett load shed was only 350 MW! (Note: that's in addition to all the generation that disconnected itself as a direct result of the voltage transients caused by the grid and generation faults/switching.) NG do say they will be looking at this area, but not at all urgently.
Logged
broadgage
Transport Scholar
Hero Member
******
Posts: 2811



View Profile
« Reply #43 on: September 11, 2019, 01:01:33 pm »

Over reliance on complicated and unreliable software is known as progress, in the railway industry and elsewhere.

I fail to see WHY a train needs to shutdown at all due to a brief and entirely foreseeable drop in frequency, and as for taking ten minutes to re-start, that is very poor design. Not being able to re-start at all without the attendance of a specialist carrying a laptop is beyond poor design, it is IMO a significant design fault.
Logged

A proper intercity train has a minimum of 8 coaches, gangwayed throughout, with first at one end, and a full sized buffet car between first and standard.
It has space for cycles, surfboards,luggage etc.
A 5 car DMU is not a proper inter-city train. The 5+5 and 9 car DMUs are almost as bad.
Electric train
Hero Member
*****
Posts: 3116


The future is 25,000 Volts a.c.


View Profile
« Reply #44 on: September 11, 2019, 07:04:25 pm »

I'm also a bit puzzled that NG say the load-shedding worked as planned, when the disconnection of 931 MW of load was offset by so much local generation being lost (disconnected by the DNO or itself) that the nett load shed was only 350 MW! (Note: that's in addition to all the generation that disconnected itself as a direct result of the voltage transients caused by the grid and generation faults/switching.) NG do say they will be looking at this area, but not at all urgently.

Mainly because these types of disruption are rare, the protection settings to react to these types of events is done is software modelling, as such its based on assumptions.

There were other disruptive weather related events happening elsewhere in the UK causing 132kV tripping's, this caused some traction rectifiers on the Mersey Rail system to lock out and signal power supply disruptions elsewhere which lead to a number of axel counters and SSI modules internal power supply crowbar to operate.

The frequency disruption was for a relatively long duration, some put this down to the lack of inertia on the National Grid due to the loss of so much heavy rotating generating plant
Logged

Mark Carne 26 June 2015 - "The challenges of delivering myriad improvement projects while still running a railway seven days a week were simply overwhelming".
Do you have something you would like to add to this thread, or would you like to raise a new question at the Coffee Shop? Please [register] (it is free) if you have not done so before, or login (at the top of this page) if you already have an account - we would love to read what you have to say!

You can find out more about how this forum works [here] - that will link you to a copy of the forum agreement that you can read before you join, and tell you very much more about how we operate. We are an independent forum, provided and run by customers of Great Western Railway, for customers of Great Western Railway and we welcome railway professionals as members too, in either a personal or official capacity. Views expressed in posts are not necessarily the views of the operators of the forum.

As well as posting messages onto existing threads, and starting new subjects, members can communicate with each other through personal messages if they wish. And once members have made a certain number of posts, they will automatically be admitted to the "frequent posters club", where subjects not-for-public-domain are discussed; anything from the occasional rant to meetups we may be having ...

 
Pages: 1 2 [3] 4
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC Valid XHTML 1.0! Valid CSS!
This forum is provided by a customer of Great Western Railway (formerly First Great Western), and the views expressed are those of the individual posters concerned. Visit www.gwr.com for the official Great Western Railway website. Please contact the administrators of this site if you feel that the content provided by one of our posters contravenes our posting rules (email link). Forum hosted by Well House Consultants

Jump to top of pageJump to Forum Home Page