How the bar for LRT trial running was lowered then lowered some more

Among the overwhelming swells of information pouring out of the Ottawa light rail public inquiry each day, there are always a few moments that rise above procurement financial details, engineers' technical talk, and political rhetoric.

On Tuesday, there was the revelation that former city transportation general manager John Manconi now works as a consultant at STV Inc., the firm that advised the city throughout the Confederation Line project and whichthe city tried to characterize asan independent third party to oversee the LRT's post-derailment return to service.

Later in the day, we heardthe LRT operator of the train that derailed leaving Tremblay station last September was communicating with the control room about the smell of human feces on the train.

That ledMichel Valo, the lawyer for train-maker Alstom,to suggest the operator was distracted and didn't notice clouds of dust and ballast flying onto the platformor the train scraping on the platform edge.

And then just when you think there's nothing more you can or want to learn about the 12-day trial run period, you find out you're wrong.

Because on Tuesday we also discovered that on top of agreeing to (and possibly even suggesting)the idea of lowering the criteria for what was supposed to be 12 consecutive days of a near-perfect simulation of full service, the city also usedits "discretion" to issue a pass on a day that should likely have been a fail.

The city lowered the bar for trial running, then lowered it again.

No straight answers running until now

Since late August 2019, it's been difficult to get straight answers about how and when the trial running that last, critical test phase of the LRT was changed.

Until now.

From the intense, frequentquestioning on the subject, it's clear the commission running the inquiry is every bit as interested in the details of the trialas we are, and possibly for reasons more than just the scoring.

• Analysis: Did the city move the goalposts for accepting LRT?

• City knowingly accepted defects with LRT and lowered trial run criteria

• It was city officials who suggested making LRT trial run testing easier, inquiry hears

Through testimony from a number of witnessesand information fromhundreds of documents posted to the inquiry's website, we've learned thecity changed its contract with consortium Rideau Transit Group (RTG) to accept 13 trains instead of 15, beginning on Aug. 16, 2019.

The next day, on Aug. 17, the city and RTG reverted to previously agreed uponcriteria from 2017 that would lower theLRT's reliability threshold.Instead of covering 98 per centof the regularly scheduled kilometres for12 straight full-service days, Confederation Line trains would only have to hit 96 per cent for nine of them.

The inquiry has heard from several witnesses that 98-per-cent reliability was an unrealistic target for a new system and that 96 per centwould still provide a good customer experience.

It was certainly unrealistic for the Confederation Line, at least in the summer of 2019. Over the 24 days of the trial run period, from July 29 to Aug. 22, the LRT scored 98 per cent only five times. (Notably, the same witnesses who've referredto the 96 per cent target as more than adequate have never mentioned the part about how the system performed terribly for one-quarter of the testing period, but still passed.)

And there's one more detail about the 2017 criteria that matters:there could be no three consecutive days below 94 per cent.

Scoring team used 'discretion' to pass 1 day

That's because the scorecardsshow there were actually four straight days where the Confederation Line did not achieve 94 per cent reliability. They were:

• Aug. 13, 2019: 91.7 per cent
• Aug. 14, 2019: 93.9 per cent
• Aug. 15, 2019: 92.3 per cent
• Aug. 16, 2019: 92.3 per cent

We've heard from witnesses that reliability around 90 per cent provides, in former RTG CEO Peter Lauch's words, a "horrendous" customer experience. So these are not results to write home about.

The middle two days the 14th and 15th were deemed "repeat" days by the trial runevaluation team, and for some reason were not countedin the consecutive day tally.

And then we come to Aug. 16. After hearing how the day was doing on a WhatsApp group chat among city transit officials and their consultants, Manconi wroteat 5:13 p.m: "We can't loose [sic] the day."

Later in the evening, the group was told there wasa train stopped due to an on-board computer problem, Manconi repeatedhis plea: "Launch a spare? Don't loose [sic] the day."

They didn't lose the day which would have extended the trial running into September. Instead, they used their discretion to issue a pass.

Decision not made in 'isolation'

Troy Charter, who was OC Transpo's representative on the trial running review team, testified Tuesday that the issue wasn't due to train issues but poor decisions by the maintenance team.

One worker decided to move a stopped train at Tunney's Pasture to the maintenance storage facility, Charter said. But it then proceeded to fail out on the main line, slowing down servicein the late afternoon and evening hours.

The other issue, said Charter, was that Alstom wasn't acting as if there were customers on those trains and was spending too much time trying to solve issues while the trains were on the track, instead of getting them off the line.

The next morning, when discussing the scoring a pass-or-fail decision was typically made by the review team the day after Charter shared the Aug. 16 score with the chat group and told them what happened.

"So we pass the day?" asked Manconi.

Charter answered that he would note the maintenance mistakes on the scorecard and "recommend we use discretion and pass the day."

When the city's lawyer asked Charter to explain the process, he said it wasn't a decision he'd make "in isolation."

"The decision of the trial run review team needs to be part of [it]," Charter said. "And we had the independent certifier as part of it."

This might sound reasonable, except for a few issues.

First, Aug. 16 was the fourth consecutive day that the reliability of the Confederation Line scored below 94 per cent, which should have triggered a restart.

Second, the fact that the score review process was overseen by an independent certifier is virtually meaningless. The certifier, as CBC reported earlier this week, is there to check off the boxes of the contract requirements, not get into the weeds of the scoring.

• Independent certifier had no input on changing scorecard for LRT trial

In an earlier discussion on the use of the city's discretion, city rail director Michael Morgan told the WhatsApp chat group that "if we want to give them a pass, the [independent certifier] may give an opinion but is unlikely to intervene."

In fact, the independent certifier team leader, Monica Sechiari of Altus Group, wasn't even at the trial testing, leaving the oversight to a junior colleague who'd only been out of college for four years.

And finally, using discretion to wave away a maintenance mistake that would have caused an actualdisruptionseems to counter the very purpose of the trial running, which is to mimic full rail service with real people.

More than just percentages

The trial run experience was about more than just percentage targets.It reflected so many of the issues being investigated as possible factors that led to early problems with the Confederation Line.

• TIMELINE |The turbulent 1st year of Ottawa's LRT network

Did the use of milestone payments in the contract put financial pressure on the builders to finish faster than they should have? Was there undue political pressure to open the system? (Mayor Jim Watson had invited a slew of VIPs to a handover ceremony on Aug. 23, 2019, two days before the trial running was completed.)

Was the maintenance team ill-equipped? Did it shoulder an unfair burden, trying to stay on top ofa system that was approved again and again, despite its documented defects? How were the corporate and public service leaders behaving in this complex and stressful partnership?

These questions are apparent in the trial running, which perhaps is why the commission has come back to those three-and-a-half weeks in the summer of 2019 again and again.

Alstom's Richard France and the city's former chief safety officer Brandon Richards are set to testify Wednesday, the penultimate day of the public hearings.