Flowing to the millions

At Valmet Automation, our control system user interface has to bring in and refine process data from various sources in near-realtime, producing inputs for rendering components that control engineers actually use to run plants and factories. This pipeline takes the form of a data flow graph that can sometimes grow close to a million nodes in size. At these scales algorithmic complexity takes a relative backseat to the physical reality of memory layout.

In this talk I'll talk you through a practical example of a "graph of objects" to "struct of arrays" transformation in the world of JavaScript, and all the fun corner cases found within!