Brought to you by TheJudge13 contributor Tourdog
Editors Note: In the absence of a clear explanation from Mercedes – F1 fans are all but happy enough to provide possible scenarios for Rosberg’s Singapore dilemma. This article was written prior to Iain:R8’s and adds additional points to consider.
Tourdog was born on the Wrong Side of the Atlantic, as he puts it, and is saddled with the strain of having to watch F1 from the perspective of an ‘Murican. His real experience lies in Audio and Tour Management, though he had an early career in computer system installation.
Never did I believe that my life experience would have any relevance to an F1 conspiracy, but here we are.
I am an end user, of what the lay term for, is a “multi pin connector”. The title is pretty self explanatory, but I shall elaborate for the uninformed. A mutli-pin connector is a quick, efficient, and relatively reliable way, to connect a large number of wires together, and take them apart. These connectors are built in a wide range of sizes, materials, and pin numbers, with each pin representing one wire in the system. I have personally assembled wiring systems not dissimilar to the ones used in racing, that had single multi pin connectors, with in excess of 150 pins, so I can speak with some authority.
ALL road cars use hundreds, if not thousands of these connectors, though most of them are a handful of pins. MOST race cars use them to make building and repair easier, though they are much stronger than the ones in the road cars, in fact a lot of the early connectors used in racing were taken right out of WWII aircraft. F1 cars use very few multi-pins now. Almost all wiring runs direct from origin to destination, through as few connectors as possible, to reduce weight. They try and limit these connections to what is necessary to quickly remove major parts, such as the steering wheel.
The steering column itself holds a “multi-Pin connector” within its hollow center, there is no coilly cable anymore like we used to see when there were only two or three buttons on the steering wheels. Since the wheel must be removed and replaced on a regular basis, the wiring has been buried within the steering column itself. One can see this from the following picture:
Unfortunately there is a line in this picture that is obscuring our view, but there appears to be about 10 pins that connect the steering wheel to the car electrically, right in the centre of the connection to the steering column. If one looks closely, they will see that the pins recessed into those 10 little holes are “male” pins. Further inspection reveals that there is an inner collar on the column connection. As the wheel is pressed onto the column, the inner collar on the steering wheel if forced backwards into the wheel by the column. This will expose the pins, allowing them to mated to the car side of the connection. The spring loaded collar is designed to protect the pins on the back of the steering wheel, so they are not damaged when the wheel is not on the car.
The “female” pins are incased in a synthetic block that resides in the canter of the steering column. There is most likely no cover for these pins, unless the teams have made a small cap that is placed over the shaft manually.
- So lets look at Mercedes statements, the key parts are highlighted:
Forensic analysis has revealed that the steering column electronic circuits were contaminated with a foreign substance. - This occurred during our normal pre-event servicing procedures at the factory and the substance found is used as part of our standard servicing procedure.
- The relevant design has been in use since 2008 (6 seasons) without experiencing any fault.
- The contamination was not visible and did not manifest itself until Sunday as Nico went to the grid, although the steering column was used throughout the weekend and the car fired up as normal on Sunday morning.
- The result was an intermittent short circuit in these circuits.
- As a consequence Nico could not command the clutch nor change engine settings.
- The car was ultimately retired because it was unsafe to execute a pit stop without command of the clutch.
- Fresh parts will be used at the forthcoming races.
- The team has been working intensively on reliability and quality processes during 2013 and 2014 in order to improve our performance in this area and these efforts will continue at the same intensive level over the coming month
The typical way to clean one of these connectors is with a quickly evaporating aerosol spray. We call it “contact cleaner”, as do you, and it can be found in any hardware or electronics store. In extreme cases however, such as an environment with exceptionally high humidity, the metal pins themselves could become corroded in a very short amount of time, as quickly as a few hours. The liquid cleaner may not have enough cutting power on its own to remove a large amount of corrosion. Keeping the pins covered, by having the wheel attached overnight, would help reduce this corrosion, as it would limit the pins exposure to the air. But in the event the wheel was left off, or even as a precautionary measure due to the humidity in Singapore, a small metal-bristled brush might be used to help clear any corrosion on the pins.
This is what could have caused the failure. If even 1 small bristle of bare wire were to break off of the brush, and lodge itself into the multi-pin connector, it could cause the failure Merc experienced. This would have been extremely difficult to see. It would also explain why the problem appeared intermittent. Depending on the length of the wire, and the layout of the pins, it could have been shorting any one, or multiple pins to each other, and/or to ground. As the wheel spun, the piece of wire could have been moving within the housing, changing the pins it touches. It could have been lodged in there since initial set-up at the factory, as they stated, though it is more likely to have been an overambitious engineer that decide to clean it, “just in case” in the garage. Simply attaching and removing the wheel could have been enough to dislodge this rogue piece of wire.
I have no doubts that this situation is entirely possible. It has happened to me. I wasn’t working on an F1 car, but I have had electrical shorts happen in this manner. Of course, there is another, equally likely situation.
Software Error.
At another point in my life, I was doing what was essentially help centre computer work, for a
completely unique, one-off software system that was responsible for getting emergency vehicles to
accident locations in a large city. Getting someone’s computer back up and running quickly, over the phone, is all about knowing what questions to ask, and the first question is always, what just changed?
This sounds overly simple, but it regularly leads to the following response:
“I just….”
Somehow even the most intelligent people think the word “just”, absolves them of responsibility. It
always means they touched something.
What tipped me off to this was Nico’s statements about the situation, they are classic defensive moves. He was very matter of fact about what he had done. He described the situation very methodically. He was in the garage. The engineers had the car up on jacks. It was running. They had been testing it and everything was fine. “The moment” he got in the car, it all went wrong. That’s the part I find hard to believe, the coincidence of timing. My first question to him would have been, “what did you touch?” I can practically hear the first words he spoke in my head, “I didn’t do anything, I just…..”
The computers and software in the car, are still very much “beta” in computer terms. Since some parts are bespoke and some of the computer modules are “standardised” parts supplied by McLaren, the hardware and the software simply cannot be tested for every possible contingency. There just isn’t enough time, and the software is updated too often. So it is entirely possible that Nico got into the car, and made a change to a setting on the wheel either he shouldn’t have, or that had not been tested correctly before. At that point the software went “haywire”. This is not a problem that only Mercedes faces, and I am sure all teams have had failures due to software at some point or another.
Why didn’t it go bad for Lewis too? Well assuming the software in both cars is identical; it could simply be a matter that Lewis didn’t make the exact same change at the same time, and he was in a different mode at the moment.
It is far better, and easier, for Merc to explain to the world that there was a “foreign substance” in the wiring, than it is to announce that they have software issues.
It was either a genuine, unforeseen electrical short, or a software error, that may have been initiated by Nico. Of course, Mercedes tagging all of their tweets with this..
#NotAConspiracy 🙂
..is probably not going to help cull the speculation.
Great article.
Maybe Mercedes needs a spray can of Caig Labs Deoxit D5 contact cleaner. This is the best stuff I have ever used and cleans up issues from audio electronics switches to doggy contacts anywhere in an automobile
So now we’re into the realm of pure speculation, right?
It’s interesting, but I think this was more likely the result of a Gremlin….or a cat.
Yeah, probably the Leprechaun took with himself a bag of a McLaren gremlins when he switched to Merc, and one just got out of the bag.. 🙂
“So now we’re into the realm of pure speculation, right?”
But what else can we do? F1 teams do not release detailed reports to the public (hell, even a 1 page report would suffice). All we get is the description of a complex event made to fit into a Twitter post. That’s worse than PowerPoint presentations!
I beg to differ… NOTHING is worse than death by PowerPoint! Ok, this is 😛
yeah dude, no worries. i was just thinking of the old Twilight Zone episode where the Gremlin almost brings down an airliner lol. creeepy 😉
Thumbsup. As far as conspiracy stories go, this is one of the rare plausible ones, for which I give you kudos 😉
Good Stuff TourDog! Thank you for this very thoughtful contribution to site site.
“Editors Note: In the absence of a clear explanation from Mercedes”
I was unaware that Mercedes or any team for that matter was required to detail exactly why any component of a car had a failure.
They’re not acquired too. Nobody said that. .
But many posters seem to believe they are
No requirement but it would look a lot less like a conspiracy if they did…
It’s a tricky one this. Those of us with a bit of relevant knowledge know that these things can happen – if you are careless – but shouldn’t happen to an organisation as well drilled for quality as an F1 team should be.
Those without the knowledge don’t have a plausible enough explanation they can accept.
That is the long and short of it, and why it keeps getting talked about in the absence of a bit more info.
@ Tourdog
Sorry – but I don’t buy the ” corrosion ” theory.
Contacts can corrode – ” in a very short amount of time, as quickly as a few hours … ”
Really ?
I’ve used electrical equipment including stuff with multi-pin connectors in far more hostile environments than Singapore without ever seeing the sort of problems you describe arising.
***
Surely Mercedes would use gold on the pins / contacts ?
Yes, you are correct, there would be gold on the contacts. The pins are plated in gold, however, and the inner pin is an alloy that is easier to solder to, and wont melt like gold will in high heat. The female pin is a single flat piece of the same gold plated alloy, that is wrapped into a circle. This forms a seam along the length of the female pin. This seam is not soldered together, so that the female side of the pin can expand and wrap around the male pin. Often the female pins edges will burr, scraping into the male pin over repeated use. If any of the underlying metal is exposed, it will corrode quickly, especially in the presence of high humidity and salt air.
Odds are, even the small amount of corrosion that MAY occur would not really have an effect on anything. However, the increased possibility due to the climactic conditions in Singapore may have led the team to clean it with some extra force, hence the brush.
I have shipped equipment into Singapore before. Signs of rust were apparent after a single day, and full-on rusting of unprotected surfaces had occurred by the time the equipment was home, a week later.
Singapore wasn’t even the worst. I could not believe the rust and corrosion on everything I shipped in and out of the Bahamas. Cars must disintegrate before your eyes there.
I can only speak from my own experiences.
Hmmm. Not sure how Nico could have caused a software error at the moment he got into the car, because at that moment the wheel wouldn’t have been on anyway, and as far as I know there aren’t any other buttons to press in modern F1 cars. Unless the theory is that Nico can cause software errors with his mind.
Simply put – he didn’t. I have this problem when dealing with my customers i.e. what’s said and what’s meant.
All Nico knows is that it was working last time he used it and now showing problems when he came to use it again. The fault could have occurred at any point during the period in between. All Nico did was discover the problem.
Thanks to Tourdog for the write up. Note not sure about the wire brush, I always use a fibre brush these days and thought it was common practise.
I’m that guy who reports real issues. Nothing changed, but the system had a failure. Shlt happens.
Man FP1 in Suzuka can’t come soon enough. Too much analysis is going into something that was a technical glitch.
Like all the analysis coming from the Hamilton fans when various things happened to scupper various races he lost out in?
Fair’s fair – if you want to go on longer than non-Hamilton fans are interested in about his problems then you’ve got to put up with other people discussing problems they are interested in.
And what does my comment has to do with Hamilton?
Just remembering what it was like wading through a similar thread after Monaco where I had similar feelings to those you expressed but held off making them public…. Part of the charm of this site it discussion isn’t suppressed.
Thank you finally some sense
Great write up. I enjoyed reading that and it makes sense. I don’t nessecarily agree, but it makes sense.
Just a sml point… Wouldn’t the issue that you suggest at the end, it NOT being a substance as Mercedes said, make it a conspiracy… I.e. A lie on Merc’s part?
So #ItIsAConspiracy
You see what you look for I guess, but with all of the criticism over the complexity of the cars this season, I could only imagine the talking points this week if Merc had announced that they had a “software failure”.
Even beyond the obvious Lewis vs Nico conspiracies, there would be the bigger issues involved with “the cars are too easy to drive” and “too many computers”, blah blah blah.
Anyway, Thank you for the support.
I don’t always agree with you either,
but I have enjoyed your contributions nonetheless.
https://pbs.twimg.com/media/By5qRkRCMAAXfUH.jpg
😀 Love it! Lol!
But yeah… so does the writer, inadvertently it seems. Ultimately, everyone is calling BS on “the substance”…
PS: You’ve rly got a hard on for the Hippo… Did he upset you that much?
From similar experiences I am convinced that the ambient humidity was the cause of the problem.
As you state these types of connector are typically cleaned/lubricated using an evaporating spray.
These sprays are mostly alcohol based. Pure alcohol is highly hygroscopic – it absorbs moisture from the atmosphere.
If an over zealous engineer had sprayed a tad more cleaner than required, it may have absorbed enough moisture that caused the cleaner not to fully evaporate (also due to the ambient humidity) and left a transparent residue with high moisture content. High moisture -> high conductivity.
IIRC from the MECU interface spec, the signals between the steering wheel and the MECU are TTL level ie: 0 to 5 volt. A value below 0.8v is considered to be 0, and above 2.0v to be 1. Just a sweaty finger on a conductor could induce an error – the probability depends on how much current is flowing through the conductor.
Marginal logic level transitions would also explain the anomalous gear changes that NR was experiencing – when switches are decoded in firmware, the signal has to be de-bounced.
When a mechanical switch operates it does not immediately change from a ON state to an OFF state, due to the contacts bouncing against the spring pressure.
To correct for this phenomenon, in firmware the position of a switch is determined by repetitively check the state until a consistent value is found. However a limit is usually imposed on how long such a check is executed to ensure an adequate response time. When the limit is reached the last value measured is used and the relevant operation (gear shift in this example) is executed. If the value is fluctuating the process may be repeated and cause a further operation to be executed.
Ironically if this was the cause of the problem, a generous blast of compressed dry air on the connector would probably have cleared the residue!
Great comment @the engineer I seem to remember you have posted some really good stuff in the past too.
Really? They use TTL signals? That surprises me for several reasons.
Not only the inherent lack of noise rejection in what must be a high EMC environment, but the fact that I’ll be surprised if anything in the ECU actually runs at 5V. Heck, these days I’d be surprised if it even runs at 3.3V, maybe the I/O but the core will more than likely be 1.8V.
I’d have assumed there would be a micro in the steering wheel and some sort of fieldbus used to connect in with the ECU but I don’t have much detailed knowledge – should have paid more attention at the seminar I attended where the head of McLaren Electronics was talking about it!
Personally, it seems more like something has caused a software glitch. What he was getting – things like 2 gear jumps – is unlikely to be caused by a poor contact.
I agree the Steering wheel electronics would be self contained communicating by a bus system to the main controller/ECU in the car. Too few pins for it to work any other way.
But could it have simply been the transceiver chip failing at the car end causing one bit to be permanently on or off. Would explain why some items worked and others failed. The fault was very permanent.
The on board camera showed that the steering wheel display was operating correctly – it is connected to the MECU by CAN bus. Obviously the other steering wheel controls are not routed via the display CAN bus, as they also would have been working correctly! In any case all the I/O on the display is used to control the display functions.
On a bussed interface all the communication is serialised, so an intermittent bus fault is unlikely to manifest itself as a consistent fault in any one single control.
@Stephen The steering wheel is bespoke to each team – McLaren Electronics would not be aware how it is designed, except perhaps for the McLaren team. Also if you were paying attention you probably would have seen the MECU interface summary which specifically mentions TTL.
Hi Eng
I’m just guessing here like everybody else.
Agree the on board display is working okay, but that is received data.
I’m assuming for the transmitted data for all the buttons, switches etc. that with the amount of data / update speed required it would be transmitted as a memory map / array (or bit map which ever you wish to call it). If one or more bits are stuck on/off strange things will happen, obviously. I’ve had similar in a very different engineering field. Of course the above could be a load of cr@p and I’m quite willing to be educated. 🙂
If you have more knowledge about the electrics / controls / ECU etc in the current cars please do an article as I would be very interested.
Hi FP Firstly this was definitely not a firmware glitch or error – the same problem was manifest with different steering wheels and also following an MECU reset.
Each time the MECU is powered up it performs the equivalent of the ‘Three fingered salute’ (crtl-alt-del). Any prior driver inputs would be reset back to default values and additionally merc have stated unequivocally that the problem was a wiring loom failure as listed in this article.
The CAN bus (Display MECU) is similar to USB – for any data to be transferred control information must pass in both directions. If there is a connection that enables it to work in one direction, it will work in the other direction. The error checking is such that the link either works or is signalled as faulty.
From the failure mode we observed, it was evident that some of the steering wheel paddles were working (intermittently), while others were not working at all and was similar on different steering wheels. This is clear evidence that the signals are not being passed via a bus architecture – hence they can only be discrete signals input directly into the MECU.
@engineer
Agreed. I think people are missing your point. The various bus types are connecting local processing and sensors for the individual car elements such as pneumatics for the gearbox, diff control, and so on.
Beautifully explained.
@tourdog
“But in the event the wheel was left off, or even as a precautionary measure due to the humidity in Singapore, a small metal-bristled brush might be used to help clear any corrosion on the pins.”
Guessing that you have never worked in the Clair Global or MSI shop. The one thing you NEVER EVER do to a connector pin, is use an abrasive material.
“This is what could have caused the failure. If even 1 small bristle of bare wire were to break off of the brush, and lodge itself into the multi-pin connector, it could cause the failure Merc experienced…….snip snip……It could have been lodged in there since initial set-up at the factory, as they stated, though it is more likely to have been an overambitious engineer that decide to clean it, “just in case” in the garage. Simply attaching and removing the wheel could have been enough to dislodge this rogue piece of wire…..snipI have no doubts that this situation is entirely possible. It has happened to me. I wasn’t working on an F1 car, but I have had electrical shorts happen in this manner.
This is not a comparatively monster size Edac or MIL-SPEC circ connector on a cross stage/FOH multi. These are small size connector systems, like a mini xlr. Remember this car was used in three practice sessions, and qualifying. Bumping and crashing over kerbs, high g stops etc., it would definitely have shown up before the start. However, if it was a badly finished crimp on the pin, with a unclamped single strand, that is a more reasonable explanation. That means bad workmanship when it was made up.
Software Error. – paragraph
That doesn’t make sense. They “rebooted”, so the ‘error’ would go. The ECU has the facility for CAN, ARCNET and FlexRay buses. Software is far from BETA, especially the ECU. The issue was multi system – gearbox, ERS, ICE, chassis systems etc. These all have their own subsystem electronics and processing.
Thank you for your input.
Perhaps you may have thought to include these arguments in your initial article.
Let me say that your derogatory opening, essentially discounting my knowledge because I don’t work for “Clair Global” most perfectly embodies why I do not work for said company. “Talking down” to others seems to be a per-requisite for everyone that works there.
And I can assure you that not working for them was my choice, as your boss asked me on several occasions. Good luck with them, you seem to fit right in.
@Tourdog
Perhaps you may have thought to include these arguments in your initial article.
They are a reply to your article. Oh I see…..
No I don’t work for Clair, but I have done consulting work for them, some while ago. The opening was not meant to be derogatory, but a coded intro. You didn’t get it. I think you are being over sensitive, the explanations are just facts as I see them. Rebut and rip them to shreds if you disagree. But you can’t deny that whiskers of wire in a connector, are just plain bad workmanship. F1 standards are ‘supposed’ to be approaching Aerospace levels. I never found Clair folks to be talking down to anybody, far from it. If people wanted to learn, they always provided the opportunity. Of course there is always the odd disagreeable person in any environment.