This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

XYZ Pointcloud reducer

tool
Credits: id144

about

Did you ever needed to reduce the size of a point-cloud file without reducing the point count?

I've made this tool which shrinks the text file by removing arbitrary amount of fraction point digits, decreasing the precision as well.

Example:

Before:
12.137491 6.127559 0.303001 48 47 50

After:
12.13 6.12 0.30 48 47 50

Usage:

XYZPointcloudReduce [-n:0..9] input_file output_file

Mandatory arguments:

input_file Input point-cloud file containing space separated values as strings. Supported formats: ASC, XYZ, etc. input_file Output point-cloud file containing space separated values as strings.

Optional arguments:

-n:0..9 Specify the count of digits after decimal point, lower number results in lower file size. Default value is 3.

Source code:

https://github.com/id144/XYZPointcloudReduce

download

xyzpointcloudreducer.zip
30.12.16 [23:21 UTC] by id144 | 1042 downloads

vasilis 16/12/2016 - 18:15

Cool!! can't open it though...it pops up a window but disappears in mil seconds..Tried to run it as an administrator but same..

id144 16/12/2016 - 20:57

@vasilis it's a console application :). use it from command line with command line arguments. i had a vvvv patch of this, but it was ugly and slow.

microdee 18/12/2016 - 05:01

now comes the 1 million dollar question for the industry stuck with these things: why on earth you want to store millions of floating point numbers in plain text human readable format? what's the problem with binary format? if you need it in a start-string/end-string situation (like json or xml or a badly written communication protocol) why not code it in base64? it always puzzled me.

andresc4 23/12/2016 - 06:11

@microdee many softs like Pix4d or agisoft give you the option to save as binary... I think X Y Z is still used in plain text because for many surveyor aplication/tools still use that format as imput, Microsft Excel is still used :D

I would like to get any other tools for heavy pointcloud visualizations in vvvv

id144 28/12/2016 - 06:29

@andresc4 converting the pointcloud into dds and visualizing it is quite straight forward. later it would be cool to have some octree system to visualize even extra large (30M+) pointclouds.

this tools was made for services that have limit on maximum file size and work only with text pointclouds. :)
obviously, you want to compress the file when possible. 7z gave me the best results.

beyon 28/12/2016 - 18:16

You should at least try to round the values instead of just truncating them, if you only use a few decimal digits the loss in accuracy could be rather significant depending on the data.

andresc4 29/12/2016 - 15:59

@if144 are you zipping rgb on one dds and xyz on a second dds , 2 filereaders and a shader like dx9 ?

id144 29/12/2016 - 23:26

@beyon sure, it does round the values, same function as FormatValue node.

1.199999 -13.575724 8.777314 158 98 75

converted to:

1.20 -13.58 8.78 158 98 75

@andresc4 Yes, FileTexture and shaders. Many DX9 particle systems actually used bitmaps/textures to store the values. This one for example ciantparticles-16000000p-gpu-particle-system- -rotations. In DX11 shader, you would use Load function instead of SampleLevel to read the values from texture. But that is for another contribution ;)

andresc4 30/12/2016 - 01:08

I was so amazed when I saw Photosynt for the first time
I remember this post of 2012 https://discourse.vvvv.org/t/photosynth-pointcloud-successfully-imported-to-vvvv/8559/13
On the end of the post I said I was about to take pictures of a huge landmark
A few years latter Pix4d was popular, this is the result
http://amdrones.com/pointcloud2/amd/monumento.html

If a webbrowser can render that... why not vvvv? well, because im doing it so wrong :P
I will give it a try latter, thanks for your help, and btw, great contribution

id144 30/12/2016 - 23:12

@andresc4
Made a small test with pointcloud data stored in DDS. Loads and renders really fast and it's easy to control pointcount too!
particles-from-dds

For reconstructions I use RealityCapture, the sample is from project Forest which we made back in 2014.

1. I've exported photogrametric reconstruction into .XYZ pointcloud
2. In my tool I've added export to two binary files (.raw) (xyz coordinates stored as Single, rgb as bytes)
3. Opened the binary files in Photoshop using RAW import
4. Saved as DDS
5. Loaded as textures and connected to a custom particle shader.

For huge pointclouds it would be best to have LOD management, something like octree. (Potree for WebGL which you posted does have octree)

beyon 31/12/2016 - 12:54

@id44 I just looked at your example numbers in your original post which looked like proper rounding wasn't used.

Also, I should have some dynamic plugins somewhere to convert plain-text point cloud files to texture inside vvvv if anyone is interested.