Extract depth maps from Google Street View

Besides high-resolution images, Google Street View also provides a depth map for each panorama, containing the distance from the camera to the nearest surface at each pixel. As far as I know, there is no official documentation of the format, but there are some open source projects which contain code showing how to interpret the data. The depth information is stored in a slightly complicated way, probably to save bandwidth. Each pixel in a grid of 512×256 pixels references one of several planes. A plane is given by its normal vector and its distance to the camera. Therefore, in order to calculate the depth at a pixel, one has to determine the intersection point of a ray starting at the center of the camera and the plane corresponding to the pixel.

The geometry of the planes and the map containing the index of the plane at each pixel can be retrieved as Base64-encoded and zlib-compressed data by requesting the following URL:


http://maps.google.com/cbk?output=json&cb_client=maps_sv&v=4&dm=1&pm=1&ph=1&hl=en&panoid=1234

I have written a small JavaScript library that fetches the data, decompresses it and computes a depth map from the planes. Here is an example:

depthmap

As usual, you can find the code on my github page.

Advertisements

Tags: , , , ,

14 Responses to “Extract depth maps from Google Street View”

  1. christian Says:

    Hi 0xef

    Thanks for this highly interesting entry! However, when I try to execute the example script from github, only the textured panorama appears and no depth-panorama is shown below as illustrated above. Did Google make any changes to the API or am I doing something wrong?

    Cheers

    • 0xef Says:

      The example script is still working, at least for me… I have tried it in recent versions of Firefox and Chrome. Did you download all files from the github repository? example.html references the files in ./js and in ../src

  2. Lois Says:

    Thx for this, so cool!

    Question: I’ve seen that you input latitude and longitude to the code, but how do i select the angle of the camera pointing in a given directon?

  3. Arnaud Sahuguet Says:

    Hi Tobias,

    I am curious if you can extract the same information for “height”.
    I am looking at StreetView data as a way to get elevation data to tell people if there is a risk of flooding with their home.
    We had Sandy in New York. You also had some flooding in Germany, I think.

    regards,

    Arnaud
    (from New York City)

  4. Einar Öberg (@inear) Says:

    Nice! I have tried before to extract the depth data but without luck. Would it be possible to generate a normal-map based on the data as well?

    • 0xef Says:

      I think that’s possible, you just have to compute derivatives of the depth in both directions on the sphere and use that to calculate the normal.

  5. Boli Says:

    Hi Oxef, why can’t I download the compressed data by the upper URL?

  6. Boli Says:

    Hi Oxef,
    what should I do if I want to extract the depth map other than the example ?

  7. Jae Hyun Yoo Says:

    Hi Oxef, How do I get valid panoid?

  8. mao Says:

    hi Oxef,
    Thanks for your great contribution. I am wondering whether there are some SPI to get the longitude and latitude of each pixel?
    thanks

  9. Remi Says:

    Hi 0xef,

    I wonder if one can get the non-paranoma photo and its depth map from the data.

    Thank you so much

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: