ALLAN JAMES EDGAR

Vehicular Distributed Computer Vision System


Home Hardware Software Venture

TODO find old architecture illustrations and link/show below

TODO could use some cleanup of the writing/storytelling (bear in mind it's not a technical document)

Guess who got some new toys??? Building on top of my Raspberry Pi GPS Logger I decided I wanted to play around with a distributed computer vision system for my car. Why? A part of me wants to implement a visual odometry algorithm in it. Might that lead to a VSLAM algorithm down the line? Could it provide me with interesting datasets to run through CNNs and test object recognition leading to (totally illegal) autonomous driving? I don't know! Why don't you keep your questions to yourself, because none of these ideas even crossed my mind!

Ok, let's get into it. So my GPS logger had a RasPi 2 as a head unit, even if all it needed to do was collect and store some video streams it would still be a touch underpowered. Time to upgrade. I've got an old macbook pro lying around with a significant (enough) processor in it. Let's use that. Another benefit of this computer is that it has; 1. an ethernet jack, 2. SSD replacement for the spinning rust (HDD) so it should handle the throughput 3. a functional battery and 4. a charger that is compatible with a car cigarette lighter. These benefits, if not immediately obvious, will be explained in due course. So I flashed it with my Linux Distro du-jour and we're in good shape. We'll call this our central processing server (shortened to *CPS). Let's look at benefit number one first. The ethernet jack. This is particularly useful as we'll be using a hardwired network in the car to get the video feeds from each of the cameras. I decided against using WIFI because a vehicle is a rather noisy electrmagnetic scenario and when it comes to correlating video feeds for locational purposes, timing is paramount.

This brings us to the next decision, what are we using to obtain the optical data. IP cameras play nicely with Linux but are too expensive. Webcams are cheap but the latency is garbage and are not always the easiest to interface with. What about our favorite single board computer? I decided to use four Raspberry Pi Zero's and four NOIR raspi camera modules with IR emitters for capturing and transmitting data. Sadly I buried the lede here because in real time the entire video feed isn't being sent to the central processing server (*CPS). Because our raspi zeros have some computation we will be extracting the "salient features" from the frames, timestamping them and sending THOSE to the *CPS. Hopefully we will record the actual footage to the SD card and send that to the CPS for storage when the car is turned off.

I hope to be able to be able to connect the CPS to my home WIFI to offload footage and data to my home storage, the possibility of this remains to be seen.

The ability to process this data on-the-fly has yet to be demonstrated so for now let's work out the network and optical architecture. In essence it's quite simple, in practicality though... We only need 5 ethernet ports, 1 for CSP and 1 each for; left, right, forward and backward facing camera units. That means that a 5 port switch will do and I found one that works off of 12V DC (same as the car's power system). Check!