Pointcloud render based on camera projection and position

[iamravisingh]

Hello folks,

I am currently working on rendering pointcloud data which are in .las format. Our current approach is to fetch the blob via a rest client and create a URL.createObjectUrl to create a unique reference such that same is used to data params to deck.
As the blobs are in GB size, network request is taking time and hampering the UX.

Now, I am thinking to change the way of rendering the points.
As the FirstPersonView is selected to visualize and render the pointcloud data, I can use the view and viewstate to get the camera position and perspective. Based on this and with bfs algo points can be rendered ( This is currently what I am thinking, not applied yet )

My questions is does anyone have ever tried this with deckgl in community? Is there any reference? Is there any othet/better approach.

Any help is appreciated. Thanks in advance

[donmccurdy]

A more scalable approach probably would be to prepare the pointcloud data in a geospatially-organized format like 3D Tiles or Cloud Optimized Point Cloud (COPC). See this demo:

https://deck.gl/examples/tile-3d-layer

If simplification of the point cloud data would be acceptable, you could also consider using libraries like meshoptimizer to reduce the scale of the data, or to produce different levels-of-detail.

[iamravisingh]

@donmccurdy Thanks for the comment.
My scenario here is to visualize the point-cloud data as a FirstPersonView. Currently, it looks like in the attached image. We are showing the parking spot of a given building or campus.

There will be no base map in my case. only a top-down image and a point-cloud data i.e. .las file. When we make an API request to pull the .las file, the request completes with 200. However, it still waits until the entire blob is downloaded and till then we show the loader. I want to change this process, was looking for something with Deckgl that is either achievable or I will have to make some changes on the service level. Say exposing an API to fetch selected byte range etc.

meshoptimizer seems promising, but I am not sure how this will help to render the dynamic .las file in the browser

[donmccurdy]

My understanding is that a LAS file must be constructed according to the COPC specification in order to support range requests directly against the file. With arbitrary LAS files, there is little a loader can do without downloading the entire file first. If you have a COPC-compliant LAS file (or similar) then I think there would be a path toward loading it in deckgl as proposed in #6816.

Alternatively, you could set up a backend service that processes the file and serves chunks of it via API call, or convert it to some other hierarchical format offline. It's likely that other formats could be more compact than LAS, but might include only visual data like position and color of each point.

[iamravisingh]

Thanks for spending time on the above query @donmccurdy. I think, for now, I will see how can I modify services that process the file and serve chunks of it via API call.

I had another concern, In my case, .las is fetched/downloaded via API call. Later, in success, we create a URL.createObjectUrl to create a unique reference such that the same is used to data params to deck.

 await getBlobByKeyVerbose(requestBuilder, layerId, pointcloudName, dispatch).then((blob) => {
            if (blob) {
                pointcloudUrl = URL.createObjectURL(blob);
            }
        });

Now assuming, we do parallel API calls and serve the .las file in chunks by using the same above approach of createObjectUrl. I am not sure whether we can append the new blob data in that existing hash URL blob:http://localhost:8000/96c06129-8ca6-48f7-98d4-68fce80e97ef. I know that in every render it creates a new reference. I just want to be sure whether parallel rendering will be possible here by passing url reference to deckgl.

[donmccurdy]

It won't be possible to modify the content associated with an existing Object URL, no — you'd need to create a new Object URL.

[iamravisingh]

As @donmccurdy said about COPC spec, I revisited my code and observed that there is already slicing happening on large dataset of .ply and then it's converted to a .las file after formatting and following some specs.

There are 2 things which I tried.

1. Tried passing the passing the location of where the file is sitting

                new PointCloudLayer({
                    id: "point-cloud-layer",
                    // data: currentPointcloudUrl,
                    data: lasUrl,
                    onDataLoad,
                    coordinateSystem: COORDINATE_SYSTEM.METER_OFFSETS,
                    getNormal: [0, 1, 0],
                    sizeScale: 15,
                    autoHighlight: true,
                    getPosition: (d: any) => d.position,
                    opacity: 1,
                    pointSize: 0.2,
                    sizeUnits: "meters",
                    pickable: true,
                    loaders: [LASLoader],
                    loadOptions: {
                        las: {
                            colorDepth: 16,
                        },
                        fetch: {
                            method: "GET",
                            header: {
                                Authorization: `Bearer ${token}`,
                            },
                        },
                    },
                    onClick: clickHandler,
                    updateTriggers: layerUpdateTrigger,
                }),
                pointLayer,
            ]);

This not working as the token is not getting passed in the header, not sure what I am missing here. As per the docs we pass loader options and I have debugged fetchFile inside deckgl is having the options that I am passing. Still the headers and not attached in net work request.

2. Tried fetching the bytes data of .las file.
   Additionally, we are using a custom API client to request to our server. I have figured out how I can get the data in bytes of the requested .las file something like this <domain>?range=bytes=92000000 i.e 92MB .las file. Now the response is in bytes and similarly, the layer is not working coz it's pointcloudlayer with LASLoader. bytes data will lose the .las file properties like x,y,z, and its RGB values. I think my only search is in loadOptions. Is there something that I can take advantage of from the load options or other properties?

 setLayers([
                new PointCloudLayer({
                    id: "point-cloud-layer",
                    data: currentPointcloudUrl,
                    onDataLoad,
                    coordinateSystem: COORDINATE_SYSTEM.METER_OFFSETS,
                    getNormal: [0, 1, 0],
                    sizeScale: 15,
                    autoHighlight: true,
                    getPosition: (d: any) => d.position,
                    opacity: 1,
                    pointSize: 0.2,
                    sizeUnits: "meters",
                    pickable: true,
                    loaders: [LASLoader],
                    loadOptions: {
                        las: {
                            colorDepth: 16,
                        },
                    },
                    onClick: clickHandler,
                    updateTriggers: layerUpdateTrigger,
                }),
                pointLayer,
            ]);

data: currentPointcloudUrl browser made reference url where the blob is stored.

At least, I was thinking 1st should work and I might play something related to batching as mentioned in docs, I was in that assumption. Now, I am kinda stuck here. I also, found a similar problem #5995

[iamravisingh]

Ah! damn, I made a terrible mistake. for Tried passing the location of where the file is sitting. loaderOptions > fetch has header and it should be headers. Now it's working, passing loadOptions to fetch is working. Although there is no difference in those, 43.2MB took 1.7min to download and render. Only difference is blob hash call is no more.

https://loaders.gl/examples/pointcloud in this example Richmond Azaelias PLY. took 21s to load 9.7MB of resources. Is showing the loader the only option, before data is downloaded?

Doing service level changes is my last resort. Before that, I want to be sure, I do not miss anything, whether things can be done in UI with the help of deckgl.
I was thinking, can we somehow play with some loader.gl parsers like batching, parseAsync?

[iamravisingh]

To be precise, is there a way to render whatever chunks are download i.e we can replace the loader and can show data as it comes in?
Tried fetching the bytes data of .las file. Can we somehow use this, is there any specific loader that will help to under stand the bytes data of las file and render it?