Extract depth maps from Google Street View

Besides high-resolution images, Google Street View also provides a depth map for each panorama, containing the distance from the camera to the nearest surface at each pixel. As far as I know, there is no official documentation of the format, but there are some open source projects which contain code showing how to interpret the data. The depth information is stored in a slightly complicated way, probably to save bandwidth. Each pixel in a grid of 512×256 pixels references one of several planes. A plane is given by its normal vector and its distance to the camera. Therefore, in order to calculate the depth at a pixel, one has to determine the intersection point of a ray starting at the center of the camera and the plane corresponding to the pixel.

The geometry of the planes and the map containing the index of the plane at each pixel can be retrieved as Base64-encoded and zlib-compressed data by requesting the following URL:


http://maps.google.com/cbk?output=json&cb_client=maps_sv&v=4&dm=1&pm=1&ph=1&hl=en&panoid=1234

I have written a small JavaScript library that fetches the data, decompresses it and computes a depth map from the planes. Here is an example:

depthmap

As usual, you can find the code on my github page.

15 thoughts on “Extract depth maps from Google Street View

  1. Hi 0xef

    Thanks for this highly interesting entry! However, when I try to execute the example script from github, only the textured panorama appears and no depth-panorama is shown below as illustrated above. Did Google make any changes to the API or am I doing something wrong?

    Cheers

    1. The example script is still working, at least for me… I have tried it in recent versions of Firefox and Chrome. Did you download all files from the github repository? example.html references the files in ./js and in ../src

  2. Thx for this, so cool!

    Question: I’ve seen that you input latitude and longitude to the code, but how do i select the angle of the camera pointing in a given directon?

  3. Hi Tobias,

    I am curious if you can extract the same information for “height”.
    I am looking at StreetView data as a way to get elevation data to tell people if there is a risk of flooding with their home.
    We had Sandy in New York. You also had some flooding in Germany, I think.

    regards,

    Arnaud
    (from New York City)

    1. I think that’s possible, you just have to compute derivatives of the depth in both directions on the sphere and use that to calculate the normal.

  4. hi Oxef,
    Thanks for your great contribution. I am wondering whether there are some SPI to get the longitude and latitude of each pixel?
    thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: