Last year we released the Enterprise PBR Material, a physically-based material that follows industry-standards and is flexible enough for use in real-time applications as well as high-quality offline rendering. The material fulfills two key requirements. The first is consistency, which means that renderings of a certain material must look the same in different applications, rendering algorithms and under different lighting conditions. The second requirement is easy-to-use, or more precise, it must be easy to build realistic-looking materials, even for inexperienced users.
The detailed scenarios and platforms for which we optimized the material were presented in a talk by Meseth et al. (2019) (video, requires login). It targets mobile phones and web browsers, VR and AR applications, real-time rasterization and ray tracing, on low-end devices and high-end workstations, but also interactive and offline global illumination rendering, on a single machine or in a cluster. In all these scenarios, we have a single source of truth, which means that the material is configured once and used everywhere without changes.
Physically-based materials and physically-based rendering naturally provide us consistency and make it possible to design an easy-to-use material model. Although the terms “physically-based material” or “PBR material” are widely used, there is a lot of misunderstanding and ambiguity. For this reason I would like to define the term “physically-based material”, show the math behind it, and describe why sticking to the fundamental, physical principles of the light transport is the key to cross-renderer consistency.
Today’s processors can do a lot of work in parallel on a single core. Programs need to be specifically designed for that, as this functionality is exposed via specialized instructions such as SSE or AVX instructions on x86 processors. By operating on wide registers storing vectors instead of scalars, a single SSE or AVX vector instruction operates on 4 or 8 numbers simultaneously.
Most programming languages do not expose vector types and functions to the programmer. Instead, compilers try to convert a scalar program into a vectorized program automatically in a process called automatic vectorization. Unfortunately, compilers often fail to do so, and it is often unclear when and why this happens.
Inspired by Nordlicht I made a small program that finds the most dominant colors in movies. Using k-means clustering, it builds a color palette with 32 colors and calculates the number of pixels with smallest distance to the cluster centers.
As it would be far too computationally expensive if clustering worked on the complete set of full-sized images of a movie, I only extracted one frame per second and scaled it down to 16×16 pixels. Moreover, I cropped black borders from each frame, and removed all logos at the beginning as well as the credits at the end. This avoids having meaningless black pixels in the clustering process.
Some results are shown in the following images. The pie charts show the colors and their frequency of occurence. The bar below shows just the colors.
‘Shared Photos’ is a small image viewer for Android that loads and displays images stored on an UPnP/DLNA server in the local network. I wrote it in the last weeks to become familiar Android development. It has some unique features which I could not find in similar apps on Google Play:
Automatic screen rotation: the image is rotated so that it covers as much pixels of the screen as possible.
Zooming, panning etc. as in the Android Gallery app.
Easy to use. Just start it, choose a server and browse your images.
I use some really great libraries in this project, namely PhotoView to display images, metadata-extractor to extract metadata/thumbnails and Cling to access DLNA servers. Thanks for that!
However, although the app uses UPnP/DLNA to communicate with the server, I can’t promise that it works with a lot of servers out there. I almost exclusively tested it with MiniDLNA 1.0.21. For this reason, I decided to not publish it on Google Play for now, maybe at some time in the future. Moreover, the app probably still contains a lot of bugs, as I have not used and tested it extensively. It basically works for me on a Sony Xperia (Android 7.0), Nexus 10 (Android 5.0), Nexus 7 2013 (Android 4.4.4) and Galaxy Nexus (Android 4.3). I will publish it here to get some feedback and because the source code may be interesting for someone.
If you want to try it out, you can download the APK below. As usual, the source is on GitHub. Feel free to fork, play with the code and contribute, if you like.
Update 2015-03-27 Added option to disable automatic screen rotation and subtitles. Update 2016-04-30 Added option to change size of thumbnails in browser. Update 2017-06-10 Save scroll position when navigating between folders (thanks to jhavens1566). Support for Android N. Update 2020-05-17 More robust DLNA browsing. Support for Android 9 and 10. Update 2020-09-27 Support for Android TV.
Besides high-resolution images, Google Street View also provides a depth map for each panorama, containing the distance from the camera to the nearest surface at each pixel. As far as I know, there is no official documentation of the format, but there are someopensource projects which contain code showing how to interpret the data. The depth information is stored in a slightly complicated way, probably to save bandwidth. Each pixel in a grid of 512×256 pixels references one of several planes. A plane is given by its normal vector and its distance to the camera. Therefore, in order to calculate the depth at a pixel, one has to determine the intersection point of a ray starting at the center of the camera and the plane corresponding to the pixel.
The geometry of the planes and the map containing the index of the plane at each pixel can be retrieved as Base64-encoded and zlib-compressed data by requesting the following URL:
This is my first attempt in real-time rendering of a procedurally generated terrain:
The terrain is generated from 3D Perlin Noise. It is converted to a signed distance field using Danielsson’s distance transform and stored in a 3D texture of size 256x256x256 on the GPU. The scene is rendered directly from the distance field, applying third-order texture filtering to improve the quality.
A straightfoward implementation of bi- or tricubic texture filtering requires lots of texture lookups. As texture lookups are slow on current graphics hardware, reducing their number greatly improves the performance of high-order filtering kernels. In contrast, linear interpolation is relatively cheap as it is supported by the hardware. This fact has been exploited by Sigg and Hadwiger (2005) to make cubic interpolation much faster by performing a series of linear texture lookups. In this article, an in-depth explanation of their technique is given, starting with some general information on linear interpolation and approximation using cubic B-splines.
WebCL is a new standard of the Khronos Group that allows web applications to benefit from the massive power of todays GPUs. It is based on OpenCL, like WebGL is based on OpenGL. Recently, I wrote a small demo that shows how WebCL can be used in image processing. I decided to implement the Edge-Avoiding À-Trous Wavelet transformation, as the algorithm is simple and gives very nice results. You need the Nokia WebCL plugin for Firefox to run the demo.