Rendered at 13:42:35 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
kasts 21 hours ago [-]
I’m not convinced GPU acceleration is a meaningful advantage for most charting use cases. Most dashboards don’t render enough data for it to matter. Once a chart is dense enough for rendering to become the bottleneck, it normally is already be too crowded to be meaningful.
Zooming can justify supporting larger datasets, but sampling/viewport culling and level of detail often avoid drawing unnecessary points...
Evidlo 20 hours ago [-]
I constantly have to work around the slowness of matplotlib when creating animated sequences for my scientific work (even with the Agg back end)
FuckButtons 11 hours ago [-]
I too have felt that pain. Funcanimation is passible for a few datapoints but it really struggles with anything meaningful in real time.
apetuskey 20 hours ago [-]
How many points are you working with usually?
all2 15 hours ago [-]
Not GP, but my datasets weigh in at around 2-20GB depending.
masenf 17 hours ago [-]
I think meaningful is in the eye of the beholder. The library is designed such that the trace buffers are directly used as inputs to the WebGL2 drawing contexts to avoid unnecessary copying throughout the stack, which does make a difference when rendering on mobile and embedded devices with limited CPU but often having GPU resources available.
apetuskey 21 hours ago [-]
It depends on how much data you are planning on showing, but as you can see from the benchmarks its also more performant than other python charting libs for small data.
We also built this library for extreme customization with CSS/Tailwind support so rendering large amounts of data is an important but not the only advantage.
mamcx 15 hours ago [-]
The good reason for worry about it is the same for data grid, list, scrolls and any other UI component that loads arbitrary data.
All UI, honestly, is only meaningful in what the screen size and our vision permit. END.
UNFORTUNATELY, you can't avoid that a user is writing "a___" and the source data has millions of things that start with `a` and all the others are dozens.
So, you can end with a massive influx of data, and sure the user see that big mess and wanna dial in, but in the meantime is nice if the UI not die in the process.
cycomanic 5 hours ago [-]
I can give you a prime example where you want to render all the data and quickly, oscilloscopes. It's quite common that you fetch traces with millions of samples, but then want to zoom into specific regions. There are lots of similar applications in experimental signal processing, where you want to have large data sets, but sampling easily will hide details that you want to see (unless you already know what exactly your data looks like).
Eridrus 10 hours ago [-]
Managing sampling is itself not totally trivial. Much easier from a DevX perspective to just have a library that can render all the datapoints.
NortySpock 13 hours ago [-]
Why settle for sampling when you can have the whole dataset?
The spiral pattern is an excellent example. "Sure it looks like this when you zoom out, but when you zoom in, you can see the finer structure of the points..."
cozzyd 10 hours ago [-]
don't you just love having a point cloud so dense that it's completely unreadable?
it seems a lot of people don't know about histograms...
genxy 17 hours ago [-]
Ok, we won't convince you and we can move on.
formerly_proven 21 hours ago [-]
There are some niche charting applications which are offloaded to FPGAs and even ASICs.
moralestapia 17 hours ago [-]
Feel free to not use it, then.
You don't have to justify your decision to people here, literally just move on with your life and forget about it.
kasts 17 hours ago [-]
I’m not trying to justify whether I'd personally use it, I was simply raising the question because the tradeoffs are interesting, and the replies, including Evidlo’s, have already taught me something.
I tried the library before commenting, it’s a cool project. The performance improvements at larger scales I've found are real. I was mainly just trying to have a conversation to learn where we all learn!
“Just move on” seems like a curious response on a discussion forum, though :)
moralestapia 16 hours ago [-]
Oh, no worries. I thought you were another one of those naysayers, of which this site has far too many already. Hehe.
Btw, English isn't my first language, so I still struggle with it sometimes. Could you point me to the part of your comment where you asked that question? I can't seem to find it. Thanks!
kasts 16 hours ago [-]
No worries!
It was not a explicit question where its easy to point at like a question mark (?) but I'd find any comment on a forum like environment to be trying to contribute to a discussion as a whole, where we can all share thoughts and counter thoughts constructively.
Anywhoo, XY seems like a cool lib. If you can find the usecase where you actually can leverage the power you should! Good job on Reflex.
ahns 19 hours ago [-]
Interesting; how do the examples compare to datashader?
Edit: for my use cases, I use napari (~1e7-8 points) if I need true interactivity; otherwise, datashader/holoviz, or even just fast-histogram's 2D histograms work.
For extremely large point clouds, these caveats[0] still apply. It irks me when people make dense scatterplots without any indication of just how dense some portions are.
Still, if it can indeed handle 1e10 points, that's pretty impressive.
I can imagine this useful to 'compress' gigabytes of data onto a 2d canvas quickly. For that, I appreciate the effort.
One thing that would be useful is to read up on Ed Tufte's principles of data visualization. Many graph libraries don't implement basic visualization principles to make they key point clear, easy to see while still keeping the full depth and complexity of data visible.
apetuskey 16 hours ago [-]
Yeah lots of things need to be improved this is the first release. Thanks for the tip
adhami 21 hours ago [-]
it's possible to render data out-of-core with XY, allowing it to render the entirety of OpenStreetMaps (that's 10,742,674,832 nodes!) with sub-second pan/zooms. it's a bit difficult to host online but you can try it out locally: https://github.com/reflex-dev/xy/tree/main/examples/osm
hantusk 17 hours ago [-]
Check out mosaic from uwdata which works on top of Observable plot
Or plotly-resampler which works on top of plotly and uses the rust package tsdownsample to aggregate on the 4pixels per pixel shown level (to make antialias work)
the grammar of graphics approach really is a great abstraction, and I'd love to see xy work in that direction
apetuskey 17 hours ago [-]
Thanks! XY already uses a similar pixel-aware approach
long line and area traces are reduced in Rust using M4 to produce viewportsized extrema, that is then refined as you zoom. Dense scatter plots use a fixed-size density grid plus a representative sample
HoneySpoons 19 hours ago [-]
This is awesome, can easily see this becoming a standard library. Can't wait for the 3D and volume visualizations.
rossant 15 hours ago [-]
See also Datoviz for 3D support in addition to fast, scalable, GPU-based 2D visualization: https://datoviz.org/ (I'm the main developer)
raychis 17 hours ago [-]
Interesting approach to large scale visualisation. Moving reduction into Rust and sending screen bounded data to WebGL seems much more sensible than pushing millions of raw points into the browser. How does it perform with real time updates? I am assuming it is much more performant? Any plans for a prod deployment?
apetuskey 17 hours ago [-]
Yes, it’s significantly faster than existing Python charting libraries for real-time updates.
Instead of serializing and sending the full dataset as JSON, it sends compact typed binary buffers and only the screen-relevant data reducing payload size and browser-side work.
We also built this library for extreme customization with CSS/Tailwind support so rendering large amounts of data is an important but not the only advantage.
All UI, honestly, is only meaningful in what the screen size and our vision permit. END.
UNFORTUNATELY, you can't avoid that a user is writing "a___" and the source data has millions of things that start with `a` and all the others are dozens.
So, you can end with a massive influx of data, and sure the user see that big mess and wanna dial in, but in the meantime is nice if the UI not die in the process.
The spiral pattern is an excellent example. "Sure it looks like this when you zoom out, but when you zoom in, you can see the finer structure of the points..."
it seems a lot of people don't know about histograms...
You don't have to justify your decision to people here, literally just move on with your life and forget about it.
I tried the library before commenting, it’s a cool project. The performance improvements at larger scales I've found are real. I was mainly just trying to have a conversation to learn where we all learn!
“Just move on” seems like a curious response on a discussion forum, though :)
Btw, English isn't my first language, so I still struggle with it sometimes. Could you point me to the part of your comment where you asked that question? I can't seem to find it. Thanks!
It was not a explicit question where its easy to point at like a question mark (?) but I'd find any comment on a forum like environment to be trying to contribute to a discussion as a whole, where we can all share thoughts and counter thoughts constructively.
Anywhoo, XY seems like a cool lib. If you can find the usecase where you actually can leverage the power you should! Good job on Reflex.
Edit: for my use cases, I use napari (~1e7-8 points) if I need true interactivity; otherwise, datashader/holoviz, or even just fast-histogram's 2D histograms work.
For extremely large point clouds, these caveats[0] still apply. It irks me when people make dense scatterplots without any indication of just how dense some portions are.
Still, if it can indeed handle 1e10 points, that's pretty impressive.
[0]: https://datashader.org/user_guide/Plotting_Pitfalls.html
One thing that would be useful is to read up on Ed Tufte's principles of data visualization. Many graph libraries don't implement basic visualization principles to make they key point clear, easy to see while still keeping the full depth and complexity of data visible.
Or plotly-resampler which works on top of plotly and uses the rust package tsdownsample to aggregate on the 4pixels per pixel shown level (to make antialias work)
the grammar of graphics approach really is a great abstraction, and I'd love to see xy work in that direction
long line and area traces are reduced in Rust using M4 to produce viewportsized extrema, that is then refined as you zoom. Dense scatter plots use a fixed-size density grid plus a representative sample
Instead of serializing and sending the full dataset as JSON, it sends compact typed binary buffers and only the screen-relevant data reducing payload size and browser-side work.
More detail here https://github.com/reflex-dev/xy#how-it-works
Love rust as the impl.
> XY is an extremely fast, interactive, customizable Python charting library
which is it?