Ken Shirriff has posted an article looking inside the ferro-magnetic chip and explaining how it works, here: www.righto.com/2024/09/ramtron-ferroelectric-fram-die.html . Not the one that failed unfortunately, as that one was lost during the first decapping attempt. It's unlikely we'd have seen anything wrong visually anyhow.
Hello Marc! I'm the national sales manager for Emco (and I'm an avid follower of your channel) if you will ever need spares for your lathe, just let me know!
The way that quadrature signal is derived from the ball bearings is amazing! Operating at 1kHz you should be able to diagnose if the signal input is working by putting a stethoscope on the transducer and listening for the 1kHz tone!
Tracking down, reading and understanding a patent, to end up replacing a memory chip ;-) Fascinating video!! Thank you. Good on you for using that clue, of losing configuration. That sure was a rabbit hole.
I can't believe how good of an idea it was to use ball bearings - that's basically an off the shelf part you can get in a tolerance of +/-0.00005". So you don't need to do much precision work yourself; no printing tiny optical scales onto glass, etc.
That’s the idea, affordable stable precision that’s oblivious to dirt and oil. To cap it off they have a pressure adjusting screw at the end of the rod so the total length of the ball stack is exact. It’s factory adjusted of course.
Would be interesting to read back the old chip and see if it's whole chip or just a stuck bit failure. Regardless, I would have assumed they store a checksum of sorts to prevent bizarre failures like this. Apparently not. Great repair as usual!
I would also like to see this. I've repaired a number of C80 DROs and the most common problem was failed keypads where the units would no longer register button presses. @curiousmarc if your keypad ever fails, I can send you a custom PCB replacement. It's a bit of work to get the old flexible one out but the replacement works very well. Another issue I had with two units, similar to what is shown in this video, where one or more axis would show corrupted data, was caused by issues with one of the socketed ICs making bad contact. Usually you could apply pressure to the affected one or slightly bend the board and it would fix it. Re-socketing each IC and cleaning the pins would fix it long term, even though they showed no signs of corrosion. I'd inspect the solder joints on the digital board as well and reflow any that seemed even a little off but I don't think it was a solder joint problem.
@@ourplesoop I have a C 80 that is losing its accuracy, I have reversed the heads and originally it was short by .020 in. but after reversing it it’s long by .005 in. Do you think by re-setting ships this may help this problem?
22:50 Thank you so much for measuring the size of the balls, I was trying to eyeball it earlier in the video and you saved me from having to go back and check
Those scales are very similar in operation to the Sony Magna-Syn scales that we still use on the larger CNC machines. Sony always provided a "converter box" to convert the "magnetic" signals to conventional TTL square wave output or the newer 1 volt peak-to-peak sine wave to allow us to connect them to any industry standard controller. Merci pour le video Jean-Marc !!!
There is a replacement type of the FRAM FM24C64B, if the WP Pin (write protection) is not used, the replacement is 1:1 compatible. If WP is used, then there are some Changes! B Type memory is fully write protected, the old type is partialy write protected only 1800h to 1FFFh is protected. If this feature is used, then FW changes must be done, in this case not possible.
A great comeback from an easy-to-make mistake. Public service achieved! Thanks for including the probe slip story. It happens so easily and can be so devastating. It's one of the most frustrating things to break something during an attempt at fixing it. I'm glad you had a replacement chip and that the FPGA was OK.
So, they made a linear resolver. Given that the resolver was the 'go to' rotary position encoder of the 70s (when first Newell patent was filed) it sort of makes sense they were thinking along those lines. The execution is brilliant. Makes me wonder about the transducers reading the bearings. Perhaps they are just mono compact cassette heads or something similar purpose made.
What’s really clever is the adaptive re-use of the ball bearings, which by necessity all have to be manufactured to super-low tolerances, and any variance with one is going to be averaged out.
I was also contemplating the nature of the odd failure. Since it displayed 'Start' on the top DRO (probably from ROM) and gibberish on the lower (instead of versions #), I wonder if it was trying to read a delimited string from the FRAM and getting back garbage which just happened to never be the delimiter.
Marc! I feel your pain... I was repairing an audio mixer, and I plugged in a molex power supply plug off by one terminal, and shorted the +48 phantom power input to the +15 volt rail. This of course blew up every single Op-Amp on the board for every channel, every bus, and every output!! That really hurt!
That is so cool! Thank you for explaining how this worked. You know, funny thing, but I did that same stupid thing on a board I was working on last week... I bridged +12V to the logic +5V and blew up the Arduino microcontroller up. I experienced the horror the moment you realized what happened... I'm glad you got lucky and didn't have more damage, or needed that PLA chip.
Nothing i enjoy more than watching along and pretending i understand what's going on 😆... One of the few content creators i don't skip through the adverts because you deserve every penny of ad revenue.
This was a particularly satisfying repair. I enjoyed seeing it on a modern piece of equipment because it's easier to follow the test process on a circuit board than on something with a tangle of wires (although watching those get untangled is its own form of entertainment!)
A former employer was manufacturing an evaluation boards for their processor cores. The CPU was on a card. Sometimes it was a hardcore (rarely), sometimes a softcore in a big ass FPGA (most common) or two for the most complex designs. We're talking about the biggest FPGAs money could buy which were expensive so dual FPGA variants were rare. The evaluation board was mostly a glorified I/O board, flash, voltage converters, what not. So there was little motivation to upgrade or in fact do any modification to board. The core of the I/O was an Intel PIIX. Eventually Intel cancelled that product line. No problem, we had ample stock. Then eventually RoHS came to truly finish of the board. The PIIX was not available in a RoHS variant, our stock was running low. The board could probably have lived another five or ten years as a meaningful evaluation platform with good software support.
My friend's company had to redesign one of their remote sensor boards when RoHS came in because the solar/battery controller chip wasn't available as lead free. The irony is that the battery was a YUASA sealed lead-acid battery which had considerably more lead than that tiny chip ever would, but "them's the rulez".
@@TheOwlman RoHS still makes sense. People have learned to recycle or dispose of lead-acid batteries properly. Here in Germany there's a significant refund on car batteries, 20€ I think it is. But electronics may still end up getting tossed out with household or other waste. And only too many amateurs or small shops only doing occasional solder work don't have proper ventilation to protect workers from lead fumes.
@@ralfbaechle I cannot disagree, millions of chips with a tiny bit of lead is still a lot of lead, it is just the irony of several kilos of it no longer being able to be charged by a chip with milligrams that struck me. It more shines a light on the past poor performance of electronics recycling that makes me hope they have improved in the last 30 years. The rise of lithium technology concerns me - I seriously doubt that will be as efficiently recycled as lead-acid until such time as either shortages or legislation make it happen. In the UK at least, the number of problems with fires from disposable vapes in domestic rubbish in on the rise; in Lincolnshire we have had a fire at a refuse facility in the last few days and a bin lorry caught fire out in the road last year, both the result of vapes.
And still there is a very toxic organic lead compound added to the fuel for airplane piston engines. Exhaust gets spilled as aerosol over our heads. How delicious. I am a bit unsure about ROHS. Devices using Lead-free solder seem to have a shorter lifespan. Alkaline batteries without added mercury will unavoidably leak and kill the device around them beyond repair, leading to even more e-waste. Batteries can even leak before they are electrically exhausted. So it's hard to notice.
If I remember correctly, there was a short time in the 1980s when alkaline batteries had become affordable and not leaking before the ban of mercury. Mercury stops a chemical process that generates gas inside the battery. Gas pressure builds up and causes the leak of potassium hydroxide solution.
Brilliant work, Marc, as usual. That method of precision linear measurement is so elegant in its simplicity. I would suspect that the ball bearings are dimensionally very high precision parts, being about as close to a consistent 10mm diameter. I bet linear position sensor assembly is pretty expensive by itself, even though it's a relatively simple assembly. I would have never guessed that they would have used a FRAM device, but perhaps they were thinking about long-life versus SRAM devices with a built-in battery, like those used for configuration memory on old Sun workstations and servers. When the battery goes flat, they won't boot. Replacing the device doesn't fix the problem, either. You have to go through manual processing of reloading the base configuration parameters using the built-in Forth interpreter. Fortunately, it was much less cumbersome to reset the config once the FRAM device was replaced. FRAM was a pretty neat technology, but as this case shows, it simply couldn't live up to the longevity of good old-fashioned magnetic core memory. The core memory in some of my old calculators from the mid-1960's are still working just as well today as when they were new. I always love watching your videos, and this one was especially fun to watch as you reasoned your way through figuring out how it worked. Kudos! Thank you for doing your videos. They are a great example of the true goodness that the Internet can offer.
Some embedded devices fall into a "crazy mode" if some internal test has failed, in this case, the non-volatile memory. The device was deliberately generating random numbers. I've seen this before with a thermostat that briefly got a bad voltage. The LCD segments started flashing randomly. In my case, simply resetting the device solved the problem. You took the long way around but we learned more about how it really works, so I guess that made it worthwhile.
Ouch! Did the exact same thing, ie, shorted across a multi volt reg and killed my STM32 Disc board. It only killed the USB com chip, but did make programming a pita.
You should go out a buy a book of Powerball tickets! You are one lucky fellow! Two bad chips, after sending 15 volts into the 5 volt rail!!!! Excellent repair as always, nicely done Marc!
As someone with little clue about electronics it is fascinating how you could reverse engineer it. Also how the Lathe uses ball bearings instead of a decoder is amazing. Thank you for the great video.
Thank you Marc, great trouble shooting video. This is an interesting tale. The decision one makes depends on the value of time. If the solution of replacing the chip were unknown, it would cost thousands of Dollars of an engineer’s time to find the fix, which luckily existed. Meanwhile if this were a production shop many thousands of Dollars of production would be lost while the equipment is down waiting for a fix. Replacing of the readout unit would be a bargain at only 1200 Dollars in the production shop case. Now that it is known that the fix is an inexpensive though possibly difficult to source memory chip, the answer for people who follow Marc’s footsteps is to fix rather than to replace. Much hinges on the availability of the obsolete memory chip. Those memory chips were $5 on eBay before this video went live; I’ll bet they suddenly got more expensive.
So the overall accuracy is tied to the accuracy of a bearing diameter, and subsequent error that accumulates as they are stacked. Fascinating. I suppose a sphere may be easier to manufacture to a precise tolerance, possibly by tumbling (honing) in a media.
those switched capacitor linear low pass filters are super cool! I've used an 8th order Butterworth lowpass filter in some A/D and D/A research projects! They are basically a brick wall filter at the cutoff frequency!
Ball bearings make sense, given the accuracy to which they are ground and their price/availability. This sounds very similar to the GE Accupins system used on 60's CNC machines. If you search "way back ge accupins" you'll find a cnczone post discussing how they're driven. It has links to images; the image dir is public and has more pics.
Thank you for clearing that mystery up for me! Have been very curious about how those sensor works for a long time! Did my education on CNC-machines in the mid 90's storing the programs on paper tape!
You are a genius! I very much like your attitude, your explanations and your superb videos. Pulled in by your AGC series once. Now a big fan. Well done. Again.
So Ken is not the only wizard of reverse engineering! I assume most failures of this unit will be this chip... the weak point shows itself... way to go Marc.... I salute you.
Very surprised by an FRAM fail, i was under the impression they were very reliable long term. Maybe not actually a failure of the underlying tech but some other common manufacturing defect that is only coming to light 20+ years down the line?
It is not core memory and wears out. It probably stores the integer ball offset in the non volatile memory. The unit is probably saving it continuously because the FRAM allows for fast writes and reads. As an potential bit of software laziness it probably reads the value from it before displaying the results which is a BAD THING to do with FRAM that has FINITE READ (and write) cycles before failure and should only be used to store data that is non-volatile and not a working variable. If you can check the read and write pins on the FRAM to confirm that it is accessing it continuously it will explain why many fail with that built in obsolescence design. Some fixes: Turn the unit off when not at the machine (It does remember the position anyway). Another possibility is to make an add on board that will prevent memory access when it is not moving. Keep a stock of 8pin sockets and FRAM chips on hand.
I guess the FPGA is just a kind of capture / compare timer. Today one could use the integrated micro controller peripherals to measure the phase precisely. Could be a nice hobby project.
An F-RAM chip lasting "only" 20 years is still a good quality design. Especially given that reads are "destructive", e.g. the values need to be written back after reading. The whole electronics box is quite a feat of engineering. It still workd. despite its usual operating environment (vibration, metal dust, chemically aggresive oils, big inductive loads on the same power cable...)
An interesting principle, though I am confused on one thing. Presumably, as the balls are a fixed spacing apart and repeat regularly along the sensor rail, does this mean the sensor can't actually tell which ball it is over, just its relative position as compared to a ball? So to measure longer distances you just have to keep reading often enough to know you haven't skipped over, and keep track of it incrementally?
From what i have seen on machining channels you generally zero in on the part in place. You touch off on the actual part as it's in de spindle and then call that 0. Absolute positioning in relation to the lathe is not relevant unless you would machine thousands of exact part but that would be more of a CNC job (i guess). Missing balls is probably not possible as you would have to go really fast with the carriage to skip over a ball.
You are right, it needs to count as the balls go. However there is more trickery than I told there. I think there are some coded magnets hidden in there that digitally encode absolute position.
Typically only CNC machines use absolute positioning, and even then, they don't typically use absolute position encoders. They use relative sensors, and a fixed home switch. To eliminate the accuracy and repeatability of the switch, the switch itself isn't "home", it just tells the machine it's in the "vicinity" of home. Once the switch is hit, then it waits for the reference pulse on the servo encoder (once per rotation of the servo motor) + some number of encoder pulses beyond that (which is settable in parameters so the absolute home position is adjustable), and then THAT is "machine reference position". Beyond that will be another physical switch which indicates OverTravel and shuts the servo drives down in case of something going beyond where it should be to prevent damage to the positioning ballscrew/ballnut. Once that home position is reached, all other movement is relative to that, and it's just counting encoder pulses. On a manual lathe, it's basically the same, but there's no fixed "home" position - you're redefining 0 wherever it needs to be in both axes based on measurements.
Not the same but perhaps inspiration, I have a height gauge for a surface plate that uses a set of stacked 1" balls and a 1" micrometer to accurately measure up to 24", APE microball
Trying to prove to myself how the phase shifted signal is produced... I noticed that the 4 sine waves would sum up to 0 if it weren't for the weighting done by the bearings. Haven't quite done it yet, but 2*sqrt(1-(x-2)floor((x+1)/2))^2) produces the inductance pattern (balls)
Got something to half-way at least demonstrate the phase-shift effect on desmos, zifewrpg5j is the ID. I've made some other guesses at the transfer function, and it seems to produce a phase shifting affect, as long as it's periodic and 'humpy' -- just sin(x) seems to work too -- but everything I've tried is non-linear w.r.t position and also has large amplitude changes. One interesting thing is that the parameters may need to be tuned to avoid amplitude inversions... I imagine that would make the FPGA's job harder.
I didn't know ferroelectric chips exist. As I understand it's not an serial EEPROM. I would love to see a detailed explanation on these with elevator music.
The remarkable bit is not what they do - plenty of chips can store data while not powered in various ways - but how they do it: normal chips tend to be rated for anything between a thousand and a hundred thousand writes (to the same bit), which plenty of applications can wear out rather quickly if not minding how often they write. Well, these are rated for TEN BILLION writes, which is a rather different ballgame (and all the more weird to see them fail). Actually, I'm beginning to wonder whether this thing didn't fail exactly because it WASN'T a real ferro chip just a pinout-compatible conventional fake which couldn't take being used as if it was the real thing...
Current production is cheap and easily available. Almost went with it for a small project of mine. Eventually decided to use PSRAM instead lol. Most are made to be pin-compatible with (I2C or SPI) EEPROMs I believe, and should be about as easy to use.
@@alexsuykov That's why I'm suspecting a possible fake - they could have dropped anything else in there and it would have just worked (until it failed a write)
One crazy chip, it would be an interesting device for a retro inspired computer build, I mean anything related to core memory has to have some retro cred.
They must spend a bit of money on those ball bearings to get very close tolerances. Over a long sensor these could add up to a lot. I wonder if they select them, or match them in sets to lock in some error offsetting within the group.
Exactly! You can play the same trick with a resolver, combine the two outputs, and get a 360 continuous phase shifter. As a coincidence I just repaired an odd piece of HP equipment that does just that!
Very nice repair, and great explanation of the mechanism. As far as the short, all I can say is that I really really despise those fat probe tips with the oversize ground ring just waiting to short something in the general vicinity. Give me a proper skinny probe tip any day. Lucky you only lost the CPU.
I used to have a real old electronics Guru named Ed Jeffries (RIP, Buddy) who was as awesome as you are Marc. Now I don't awe easily but - like Ed used to do - You awe the shit out out of me Marc !
It is understandable that they didn't want their tech to be put in the public domain, but they didn't count on @CuriousMarc being their adversary. It is an example of where a better strategy for keeping a secret is helping the guy who doesn't know the secret but is clever enough to figure it out anyway; would he have made this video if the vendor provided the fix? That is a question for Marc.
What a goofy repair. Reminds me of HP laptops that scramble their bios contents on every power cycle and they eventually kill the memory cells in the chip. Side note, I'm so glad the overvoltage repair was simpler than the HP 9825 repair saga!
Wow. I'd love an elevator music version of how that FRAM works, since it doesn't seem like EEPROM I'm used to. Also, I do get worried the more modern unserviceable tech we get with no manuals and eventually no spare parts available we're gonna need more and more general "reverse engineering" skills. Bit of a worry though buying parts of unknown quality though since I've certainly ended up with fakes before.
I love your work, starting from reverse engineering (wish i could do similar things) and also figuring out how things work. Did not expect such a solution. Still one question is open to me, it measures only one time the 10mm and then over and over right? Means it is counting then for ongoing measures. great job!
Does this have a second channel of excitation with 90 degree phase difference then? I've been messing with resolvers at work, and they have one channel of excitation and two outputs for sine and cosine. If this only has one output it must have two quadrature inputs.
the thing with the phase and multiple channels... would that be how digital calipers work? timing the phase and counting beats from the capacitive finger assembly?
Actually, I'll make a confidence, I've always refused to by digital calipers because I have no idea how they work and I'm too afraid they use a crappy system for reading the measurements. However, after seeing this video, I'll get more information, and maybe change my mind!
Ken Shirriff has posted an article looking inside the ferro-magnetic chip and explaining how it works, here: www.righto.com/2024/09/ramtron-ferroelectric-fram-die.html . Not the one that failed unfortunately, as that one was lost during the first decapping attempt. It's unlikely we'd have seen anything wrong visually anyhow.
That first version of the reverse engineered schematic is worth a print on a t-shirt 👍
I'd buy that.
I would buy that.
take my money
You went down the rabbit hole and came out with the rabbit and hat , well done. this should help other owners too.
Beautiful clean Sensor Signal, well deserved patent as it is even mechanical pretty accurate and still simple to process
Hello Marc! I'm the national sales manager for Emco (and I'm an avid follower of your channel) if you will ever need spares for your lathe, just let me know!
@@federicos8504 Woohoo! I love that lathe. Thanks for posting here!
The way that quadrature signal is derived from the ball bearings is amazing! Operating at 1kHz you should be able to diagnose if the signal input is working by putting a stethoscope on the transducer and listening for the 1kHz tone!
Tracking down, reading and understanding a patent, to end up replacing a memory chip ;-)
Fascinating video!! Thank you.
Good on you for using that clue, of losing configuration. That sure was a rabbit hole.
I think I got lucky twice on that repair…
@@CuriousMarc You sure did, the epic HP repair from a few years ago was caused by a similar short and it took months.
I can't believe how good of an idea it was to use ball bearings - that's basically an off the shelf part you can get in a tolerance of +/-0.00005". So you don't need to do much precision work yourself; no printing tiny optical scales onto glass, etc.
That’s the idea, affordable stable precision that’s oblivious to dirt and oil. To cap it off they have a pressure adjusting screw at the end of the rod so the total length of the ball stack is exact. It’s factory adjusted of course.
Would be interesting to read back the old chip and see if it's whole chip or just a stuck bit failure. Regardless, I would have assumed they store a checksum of sorts to prevent bizarre failures like this. Apparently not. Great repair as usual!
I would also like to see this. I've repaired a number of C80 DROs and the most common problem was failed keypads where the units would no longer register button presses.
@curiousmarc if your keypad ever fails, I can send you a custom PCB replacement. It's a bit of work to get the old flexible one out but the replacement works very well.
Another issue I had with two units, similar to what is shown in this video, where one or more axis would show corrupted data, was caused by issues with one of the socketed ICs making bad contact. Usually you could apply pressure to the affected one or slightly bend the board and it would fix it. Re-socketing each IC and cleaning the pins would fix it long term, even though they showed no signs of corrosion. I'd inspect the solder joints on the digital board as well and reflow any that seemed even a little off but I don't think it was a solder joint problem.
@@ourplesoop
I have a C 80 that is losing its accuracy, I have reversed the heads and originally it was short by .020 in. but after reversing it it’s long by .005 in.
Do you think by re-setting ships this may help this problem?
@@lazman111 He said there is a screw compressing the ball bearings you can check. The FRAM chip won't affect the accuracy.
22:50 Thank you so much for measuring the size of the balls, I was trying to eyeball it earlier in the video and you saved me from having to go back and check
Two geniuses at play here, the person who designed this and you for being able to figure out how it worked.
Those scales are very similar in operation to the Sony Magna-Syn scales that we still use on the larger CNC machines. Sony always provided a "converter box" to convert the "magnetic" signals to conventional TTL square wave output or the newer 1 volt peak-to-peak sine wave to allow us to connect them to any industry standard controller. Merci pour le video Jean-Marc !!!
There is a replacement type of the FRAM FM24C64B, if the WP Pin (write protection) is not used, the replacement is 1:1 compatible. If WP is used, then there are some Changes! B Type memory is fully write protected, the old type is partialy write protected only 1800h to 1FFFh is protected. If this feature is used, then FW changes must be done, in this case not possible.
Thanks for the tip!
A great comeback from an easy-to-make mistake. Public service achieved! Thanks for including the probe slip story. It happens so easily and can be so devastating. It's one of the most frustrating things to break something during an attempt at fixing it. I'm glad you had a replacement chip and that the FPGA was OK.
So, they made a linear resolver. Given that the resolver was the 'go to' rotary position encoder of the 70s (when first Newell patent was filed) it sort of makes sense they were thinking along those lines. The execution is brilliant. Makes me wonder about the transducers reading the bearings. Perhaps they are just mono compact cassette heads or something similar purpose made.
What’s really clever is the adaptive re-use of the ball bearings, which by necessity all have to be manufactured to super-low tolerances, and any variance with one is going to be averaged out.
I was also contemplating the nature of the odd failure. Since it displayed 'Start' on the top DRO (probably from ROM) and gibberish on the lower (instead of versions #), I wonder if it was trying to read a delimited string from the FRAM and getting back garbage which just happened to never be the delimiter.
Marc! I feel your pain... I was repairing an audio mixer, and I plugged in a molex power supply plug off by one terminal, and shorted the +48 phantom power input to the +15 volt rail.
This of course blew up every single Op-Amp on the board for every channel, every bus, and every output!! That really hurt!
Even reading about this hurts...
A lot of "penalty" hours with that desk 😅
Ain’t that annoying when that happens. But it is our lot. We repairers are made to suffer from blowing up our own parts.
That is so cool! Thank you for explaining how this worked. You know, funny thing, but I did that same stupid thing on a board I was working on last week... I bridged +12V to the logic +5V and blew up the Arduino microcontroller up. I experienced the horror the moment you realized what happened... I'm glad you got lucky and didn't have more damage, or needed that PLA chip.
It’s part of the occupation hazards…
I always wondered how those linear measurement tracks work. Thanks for that
Nothing i enjoy more than watching along and pretending i understand what's going on 😆... One of the few content creators i don't skip through the adverts because you deserve every penny of ad revenue.
This was a particularly satisfying repair. I enjoyed seeing it on a modern piece of equipment because it's easier to follow the test process on a circuit board than on something with a tangle of wires (although watching those get untangled is its own form of entertainment!)
A former employer was manufacturing an evaluation boards for their processor cores. The CPU was on a card. Sometimes it was a hardcore (rarely), sometimes a softcore in a big ass FPGA (most common) or two for the most complex designs. We're talking about the biggest FPGAs money could buy which were expensive so dual FPGA variants were rare. The evaluation board was mostly a glorified I/O board, flash, voltage converters, what not. So there was little motivation to upgrade or in fact do any modification to board. The core of the I/O was an Intel PIIX. Eventually Intel cancelled that product line. No problem, we had ample stock. Then eventually RoHS came to truly finish of the board. The PIIX was not available in a RoHS variant, our stock was running low. The board could probably have lived another five or ten years as a meaningful evaluation platform with good software support.
My friend's company had to redesign one of their remote sensor boards when RoHS came in because the solar/battery controller chip wasn't available as lead free. The irony is that the battery was a YUASA sealed lead-acid battery which had considerably more lead than that tiny chip ever would, but "them's the rulez".
@@TheOwlman RoHS still makes sense. People have learned to recycle or dispose of lead-acid batteries properly. Here in Germany there's a significant refund on car batteries, 20€ I think it is. But electronics may still end up getting tossed out with household or other waste. And only too many amateurs or small shops only doing occasional solder work don't have proper ventilation to protect workers from lead fumes.
@@ralfbaechle I cannot disagree, millions of chips with a tiny bit of lead is still a lot of lead, it is just the irony of several kilos of it no longer being able to be charged by a chip with milligrams that struck me. It more shines a light on the past poor performance of electronics recycling that makes me hope they have improved in the last 30 years. The rise of lithium technology concerns me - I seriously doubt that will be as efficiently recycled as lead-acid until such time as either shortages or legislation make it happen. In the UK at least, the number of problems with fires from disposable vapes in domestic rubbish in on the rise; in Lincolnshire we have had a fire at a refuse facility in the last few days and a bin lorry caught fire out in the road last year, both the result of vapes.
And still there is a very toxic organic lead compound added to the fuel for airplane piston engines. Exhaust gets spilled as aerosol over our heads. How delicious.
I am a bit unsure about ROHS. Devices using Lead-free solder seem to have a shorter lifespan.
Alkaline batteries without added mercury will unavoidably leak and kill the device around them beyond repair, leading to even more e-waste. Batteries can even leak before they are electrically exhausted. So it's hard to notice.
If I remember correctly, there was a short time in the 1980s when alkaline batteries had become affordable and not leaking before the ban of mercury. Mercury stops a chemical process that generates gas inside the battery. Gas pressure builds up and causes the leak of potassium hydroxide solution.
Brilliant work, Marc, as usual. That method of precision linear measurement is so elegant in its simplicity. I would suspect that the ball bearings are dimensionally very high precision parts, being about as close to a consistent 10mm diameter. I bet linear position sensor assembly is pretty expensive by itself, even though it's a relatively simple assembly. I would have never guessed that they would have used a FRAM device, but perhaps they were thinking about long-life versus SRAM devices with a built-in battery, like those used for configuration memory on old Sun workstations and servers. When the battery goes flat, they won't boot. Replacing the device doesn't fix the problem, either. You have to go through manual processing of reloading the base configuration parameters using the built-in Forth interpreter. Fortunately, it was much less cumbersome to reset the config once the FRAM device was replaced. FRAM was a pretty neat technology, but as this case shows, it simply couldn't live up to the longevity of good old-fashioned magnetic core memory. The core memory in some of my old calculators from the mid-1960's are still working just as well today as when they were new. I always love watching your videos, and this one was especially fun to watch as you reasoned your way through figuring out how it worked. Kudos! Thank you for doing your videos. They are a great example of the true goodness that the Internet can offer.
Some embedded devices fall into a "crazy mode" if some internal test has failed, in this case, the non-volatile memory. The device was deliberately generating random numbers. I've seen this before with a thermostat that briefly got a bad voltage. The LCD segments started flashing randomly. In my case, simply resetting the device solved the problem. You took the long way around but we learned more about how it really works, so I guess that made it worthwhile.
Again, thanks for showing your mistakes as well. We learn as much from these as from your (often incredible) successes.
Ouch! Did the exact same thing, ie, shorted across a multi volt reg and killed my STM32 Disc board. It only killed the USB com chip, but did make programming a pita.
As usual, I’m in awe of the combination of comprehension and execution…wow! 👍👍
You should go out a buy a book of Powerball tickets! You are one lucky fellow! Two bad chips, after sending 15 volts into the 5 volt rail!!!! Excellent repair as always, nicely done Marc!
Merci for the video! I am now a little more appreciative of my Fanuc controllers at work.
As someone with little clue about electronics it is fascinating how you could reverse engineer it. Also how the Lathe uses ball bearings instead of a decoder is amazing. Thank you for the great video.
It would be interesting to test the FRAM chip outside, and see if it is a single bit failure, all of it, or you just get random bits out of it.
Yup, learned something (several things) fun about electronics again from Prof Marc.
Thanks for taking up this noble cause and succeeding ! ... and ... I appreciate the learning.
I always wondered how they worked, such a simple but genius way of doing it! Fabulous and thank you.
Great video! Very cool how the sensor works. Thanks!
I'm impress, the vernier thing with the phases reading is a very clever idea. And that type of memory i never know about it.
Great episode.
And I am even more astonished when I see that the people who comment know more about the topics presented than I do. I am truly in the right place.
Amazing technology. Very ingenious! Thanks
It's easy when you understand and know what you're doing!
Nicely done! I too was thinking a bad crapacitor, but I would have never guessed the fault would be in the FRAM.
Thank you Marc, great trouble shooting video. This is an interesting tale. The decision one makes depends on the value of time. If the solution of replacing the chip were unknown, it would cost thousands of Dollars of an engineer’s time to find the fix, which luckily existed. Meanwhile if this were a production shop many thousands of Dollars of production would be lost while the equipment is down waiting for a fix. Replacing of the readout unit would be a bargain at only 1200 Dollars in the production shop case.
Now that it is known that the fix is an inexpensive though possibly difficult to source memory chip, the answer for people who follow Marc’s footsteps is to fix rather than to replace.
Much hinges on the availability of the obsolete memory chip. Those memory chips were $5 on eBay before this video went live; I’ll bet they suddenly got more expensive.
So the overall accuracy is tied to the accuracy of a bearing diameter, and subsequent error that accumulates as they are stacked. Fascinating. I suppose a sphere may be easier to manufacture to a precise tolerance, possibly by tumbling (honing) in a media.
those switched capacitor linear low pass filters are super cool! I've used an 8th order Butterworth lowpass filter in some A/D and D/A research projects! They are basically a brick wall filter at the cutoff frequency!
Ball bearings make sense, given the accuracy to which they are ground and their price/availability.
This sounds very similar to the GE Accupins system used on 60's CNC machines. If you search "way back ge accupins" you'll find a cnczone post discussing how they're driven. It has links to images; the image dir is public and has more pics.
glad not to be the only one doing dumb mistake during fault debug
That circuit is a very eloquent and brilliant design. Great reverse engineering Marc!
A VERY good find. Congratulations!
Excellent repair. 👌 Machine shop equipment seems to be in a world of their own. Very clever design ideas. Thank you for a great step-by-step video.
It’s good idea that you put IC socket there❤
That was particularly interesting, thanks dude!
Absolutely fascinating! thankyou Marc
Thank you for clearing that mystery up for me!
Have been very curious about how those sensor works for a long time! Did my education on CNC-machines in the mid 90's storing the programs on paper tape!
You are a genius! I very much like your attitude, your explanations and your superb videos. Pulled in by your AGC series once. Now a big fan. Well done. Again.
Fantastic video. My favourite so far!
So Ken is not the only wizard of reverse engineering! I assume most failures of this unit will be this chip... the weak point shows itself... way to go Marc.... I salute you.
I’m a baby reverse engineer compared to what Master Ken and TubeTime can do!
@@CuriousMarcNone of us were born knowing this stuff, gotta start somewhere!
@@CuriousMarc Do Patreons get the diagram? That would be nice. Or better yet, release it into the wild!
@@scowell Sure! I can make it available on Patreon. Eventually, when I get to it, it'll be released on my web site.
Very surprised by an FRAM fail, i was under the impression they were very reliable long term. Maybe not actually a failure of the underlying tech but some other common manufacturing defect that is only coming to light 20+ years down the line?
I'm surprised too. I bet they used a FRAM instead of an EEPROM for reliability. Or maybe the FPGA needs to write to it frequently?
It is not core memory and wears out.
It probably stores the integer ball offset in the non volatile memory. The unit is probably saving it continuously because the FRAM allows for fast writes and reads. As an potential bit of software laziness it probably reads the value from it before displaying the results which is a BAD THING to do with FRAM that has FINITE READ (and write) cycles before failure and should only be used to store data that is non-volatile and not a working variable.
If you can check the read and write pins on the FRAM to confirm that it is accessing it continuously it will explain why many fail with that built in obsolescence design.
Some fixes:
Turn the unit off when not at the machine (It does remember the position anyway).
Another possibility is to make an add on board that will prevent memory access when it is not moving.
Keep a stock of 8pin sockets and FRAM chips on hand.
Yep that was cool. Funny how small innovations are built on over and over again. The good old Schmitt trigger comes up over and over again.;)
Well done as always
I guess the FPGA is just a kind of capture / compare timer. Today one could use the integrated micro controller peripherals to measure the phase precisely. Could be a nice hobby project.
Thanks Marc for the amazingly interesting educating entertaining video !!!
That was incredible luck with that atmel
Using ball bearings as a reference length is super interesting.
That's a mad hand written schematic!
An F-RAM chip lasting "only" 20 years is still a good quality design. Especially given that reads are "destructive", e.g. the values need to be written back after reading. The whole electronics box is quite a feat of engineering. It still workd. despite its usual operating environment (vibration, metal dust, chemically aggresive oils, big inductive loads on the same power cable...)
An interesting principle, though I am confused on one thing. Presumably, as the balls are a fixed spacing apart and repeat regularly along the sensor rail, does this mean the sensor can't actually tell which ball it is over, just its relative position as compared to a ball? So to measure longer distances you just have to keep reading often enough to know you haven't skipped over, and keep track of it incrementally?
From what i have seen on machining channels you generally zero in on the part in place. You touch off on the actual part as it's in de spindle and then call that 0.
Absolute positioning in relation to the lathe is not relevant unless you would machine thousands of exact part but that would be more of a CNC job (i guess). Missing balls is probably not possible as you would have to go really fast with the carriage to skip over a ball.
You are right, it needs to count as the balls go. However there is more trickery than I told there. I think there are some coded magnets hidden in there that digitally encode absolute position.
Typically only CNC machines use absolute positioning, and even then, they don't typically use absolute position encoders. They use relative sensors, and a fixed home switch. To eliminate the accuracy and repeatability of the switch, the switch itself isn't "home", it just tells the machine it's in the "vicinity" of home. Once the switch is hit, then it waits for the reference pulse on the servo encoder (once per rotation of the servo motor) + some number of encoder pulses beyond that (which is settable in parameters so the absolute home position is adjustable), and then THAT is "machine reference position". Beyond that will be another physical switch which indicates OverTravel and shuts the servo drives down in case of something going beyond where it should be to prevent damage to the positioning ballscrew/ballnut. Once that home position is reached, all other movement is relative to that, and it's just counting encoder pulses. On a manual lathe, it's basically the same, but there's no fixed "home" position - you're redefining 0 wherever it needs to be in both axes based on measurements.
@@CuriousMarc Try moving the lathe while the DRO is powered off. Does it register position changes even while powered down? 🤔
great repair 😁
I wonder who first had the idea for that ball bearing sensor? Simple but very clever!
Maybe it's Newall? I don't see any prior art citation in the British patent.
Not the same but perhaps inspiration, I have a height gauge for a surface plate that uses a set of stacked 1" balls and a 1" micrometer to accurately measure up to 24", APE microball
excellent
When it was broken, I wonder if that "random" number output was actually the board executing a ROM dump
Wow! It wasn't a dead battery on a Dallas battery backed RAM chip!?
That was great.
Not such a far shot from a DSKY, haha. Nice fix - good luck with the machine!
Come to think of it, it *does* look like a DSKY! Wouldn’t it be fun to make a replacement that looks exactly like one 😊
@@CuriousMarc ...where you need to enter numeric nouns and verbs to control the machine!
Would be interesting to put the memory chip in a demagnetiser to see if it could be fixed
thank you for solving the mystery of what size ball bearings are inside the sensor 😂
The electronics version of for the loss of a nail the shoe was lost etc.
Well there you go. I learned a very interesting way to do meteorology today.
"electrolytic Crapacitors". They were new for me😅
Interesting sight into a laboratory I had not yet seen.
Trying to prove to myself how the phase shifted signal is produced... I noticed that the 4 sine waves would sum up to 0 if it weren't for the weighting done by the bearings. Haven't quite done it yet, but 2*sqrt(1-(x-2)floor((x+1)/2))^2) produces the inductance pattern (balls)
Got something to half-way at least demonstrate the phase-shift effect on desmos, zifewrpg5j is the ID. I've made some other guesses at the transfer function, and it seems to produce a phase shifting affect, as long as it's periodic and 'humpy' -- just sin(x) seems to work too -- but everything I've tried is non-linear w.r.t position and also has large amplitude changes. One interesting thing is that the parameters may need to be tuned to avoid amplitude inversions... I imagine that would make the FPGA's job harder.
I didn't know ferroelectric chips exist. As I understand it's not an serial EEPROM. I would love to see a detailed explanation on these with elevator music.
It’s called an FRAM. It’s essentially a modern version of core memory.
The remarkable bit is not what they do - plenty of chips can store data while not powered in various ways - but how they do it: normal chips tend to be rated for anything between a thousand and a hundred thousand writes (to the same bit), which plenty of applications can wear out rather quickly if not minding how often they write. Well, these are rated for TEN BILLION writes, which is a rather different ballgame (and all the more weird to see them fail).
Actually, I'm beginning to wonder whether this thing didn't fail exactly because it WASN'T a real ferro chip just a pinout-compatible conventional fake which couldn't take being used as if it was the real thing...
Current production is cheap and easily available. Almost went with it for a small project of mine. Eventually decided to use PSRAM instead lol. Most are made to be pin-compatible with (I2C or SPI) EEPROMs I believe, and should be about as easy to use.
@@alexsuykov That's why I'm suspecting a possible fake - they could have dropped anything else in there and it would have just worked (until it failed a write)
Maybe evilmonkeyzdesign could decap it for a look see look see.
So smart!
One crazy chip, it would be an interesting device for a retro inspired computer build, I mean anything related to core memory has to have some retro cred.
Super cool
They must spend a bit of money on those ball bearings to get very close tolerances. Over a long sensor these could add up to a lot. I wonder if they select them, or match them in sets to lock in some error offsetting within the group.
Sounds very much like the linear version of a resolver.
Exactly! You can play the same trick with a resolver, combine the two outputs, and get a 360 continuous phase shifter. As a coincidence I just repaired an odd piece of HP equipment that does just that!
Very nice repair, and great explanation of the mechanism. As far as the short, all I can say is that I really really despise those fat probe tips with the oversize ground ring just waiting to short something in the general vicinity. Give me a proper skinny probe tip any day. Lucky you only lost the CPU.
Amazing!
Sure beats a hslf hour watching a "reality" show! This is reality stripped down to its undershorts!
I used to have a real old electronics Guru named Ed Jeffries (RIP, Buddy) who was as awesome as you are Marc. Now I don't awe easily but - like Ed used to do - You awe the shit out out of me Marc !
It is understandable that they didn't want their tech to be put in the public domain, but they didn't count on @CuriousMarc being their adversary. It is an example of where a better strategy for keeping a secret is helping the guy who doesn't know the secret but is clever enough to figure it out anyway; would he have made this video if the vendor provided the fix? That is a question for Marc.
Love it , as always😎
Why do they have to put the input and output pins on so many voltage regulators right next to each other?
My thoughts exactly! Why? Why?
What a goofy repair. Reminds me of HP laptops that scramble their bios contents on every power cycle and they eventually kill the memory cells in the chip.
Side note, I'm so glad the overvoltage repair was simpler than the HP 9825 repair saga!
Me too! I had a bad flashback...
Wow. I'd love an elevator music version of how that FRAM works, since it doesn't seem like EEPROM I'm used to. Also, I do get worried the more modern unserviceable tech we get with no manuals and eventually no spare parts available we're gonna need more and more general "reverse engineering" skills. Bit of a worry though buying parts of unknown quality though since I've certainly ended up with fakes before.
can you decap the FRAM to see how its made?
Not yet, but I kept the FRAM chip (and the 8051 microcontroller). Maybe Master Ken will be interested.
Switched-capacitor filters are not "digital", though they are discrete time.
Marc: _“…bad Chinese electrolytic _*_crapacitors…_*_ “_
Me: _That’s what I heard, right…?_ 😂
Yes. A crapacitor. That's what they are called, right?
I love your work, starting from reverse engineering (wish i could do similar things) and also figuring out how things work. Did not expect such a solution. Still one question is open to me, it measures only one time the 10mm and then over and over right? Means it is counting then for ongoing measures.
great job!
Outstanding sleuthing, and oh such a satisfying solution! You earned yourself a nice glass of scotch in your easy chair for this one!
Does this have a second channel of excitation with 90 degree phase difference then? I've been messing with resolvers at work, and they have one channel of excitation and two outputs for sine and cosine. If this only has one output it must have two quadrature inputs.
the thing with the phase and multiple channels... would that be how digital calipers work? timing the phase and counting beats from the capacitive finger assembly?
Actually, I'll make a confidence, I've always refused to by digital calipers because I have no idea how they work and I'm too afraid they use a crappy system for reading the measurements. However, after seeing this video, I'll get more information, and maybe change my mind!
2:45 We need a group that that make obsolete and jellybean parts to combat the shady Chinese obsolete component seller and continu Through-hole parts.
If not make , ethically recycle obsolete parts from old equipment.
Newark Electronics to the rescue!