Project Idea: M2 as DVR for in-vehicle cameras?

Developer Projects
#1

First post, and sharing the concept of my initial project idea for the M2. I am a $79 Final Version M2 Kickstarter backer and have a 2017 Chevy Bolt EV, with 5 cameras (one front, one on each mirror, one rear wide angle, one rear) and I’m thinking the first project I’d want to start with the M2 is a DVR mode for the cameras.

First off, the idea would be done in phases, each trying to answer simple questions…

  1. Can the M2 see the video streams from the cameras at all? Is this project even possible?
  2. Do the cameras send video data consistently? Or is there a trigger required to have them send data?
  3. Can I record/store a single video stream to the SD Card on the M2? Is the speed of the interface viable for this?
  4. Can I record/store multiple video streams to the SD Card on the M2? Will the Atmel SAM3X8E processor be able to keep up?
  5. Can I add sensor data to trigger recording? (read from the impact sensors in-vehicle or an add-on accelerometer (like ADXL337 or ADXL377) to trigger storing video as needed)
  6. Add a rolling buffer of video, so when the sensor is triggered, the previous X seconds of video is stored.
  7. Test to see how much functionality is still available when vehicle is “off” to see if can be used as a “tamper DVR”

So, I thought as I’m kicking the idea around and thinking of writing some of the sketch for this before I get the M2… Why not ask if you think the idea is viable with the M2.

Thanks,

Scott

1 Like
#2

How is the video used? Is it displayed as the surrounding area on your head unit?

You need to first track down the wiring from the camera and find out where it goes.

I suspect it goes directly to the head unit and is not available on the databus. Video is a high bandwidth stream and I highly doubt it goes over the canbus.

The good news is if it does go directly to the head unit you should be able to split the video off and process it separately and it may be a common video output. But you won’t know until you track down the wiring and see where it goes. Unless you have the OEM shop manual available.

#3

At least in the Chevy Bolt EV, the video from the cameras are displayed on the 10.2" LCD screen that is part of the head unit/heater controls/real-time display/car settings/etc… So, I really think it’s on “a” bus somewhere. The cameras are used by the lane-departure warning, blind spot detection, rear camera mirror, and forward collision alert just to pick a few systems that use them.

So, they are part of the overall integrated system in the car. Now, other vehicles may note do this, but the Bolt being a technology show case for GM…

#4

Looking at the specs on GM’s page…

I suspect they are using the same boards that I have here and were installed in the Cadillac CTS as of 2016 for their VUE system. This is a high performance ARM board designed to do the things you are enjoying.

Note that the Blind Spot detection uses radar, and the collision system says it uses a combination of radar and camera. The Pedestrian detection likely uses the camera. The lane departure is fully video based. Lots of the features included are radar based but they do have some computer vision related detection as well.

You will find that these boards have some additional hardware to process video to do everything on board.

If you look at this board here it would be similar to what is in your head unit. The only difference would be the specific support hardware it has:

https://boundarydevices.com/product/nitrogen6max/

This unit supports 2 parallel camera interfaces and 1 MIPI camera port which supports up to 4 cameras. My guess is they support up to 6 cameras. I know they support up to 4 video outputs so it is likely your cluster and center display come directly from the head unit.

You can find information about MIPI here:

The M2 will likely not be of much help here. You likely really need to be able to write an application for your head unit. I do not know what OS GM is using in its current cars though. I can install Android, Windows, Linux and probably some others onto the boards I have here.With a custom application in your head unit you might be able to send commands to the head unit to have it do things. But I HIGHLY doubt GM offers a developer program to the public to develop apps for the head units.

Rodney

#5

With GM’s Gen5/6 Camaros, the cameras output standard composite video, not any type of digital data stream

#6

I suspect that would be the case with the Cadillac Vue as well since it is cheaper to use cameras already in existence. I would be surprised if the over head view is not using Composite video feeding to a convertor that converts to the mipi interface. I could be wrong however so you may want to chase down the wiring. I know for the Nav system in my '09 CTS they use standard composite but uses standard wire and not coax to hook it up.

But either way your not likely going to use a RPI to record video from all of the cameras at the same time.

#7

You would need an mpg encoder for each camera to get the data sent digitally. There is software for rPi and other similar computer boards to do motion tracking on encoded composite video (which is quite small compared to 720/1080/4K video streams). You could track from several streams simultaneously.