Thanks to the use of cloud infrastructure, clusterized resources and dockerized services we have built a dynamic infrastucture that would have been very hard to achieve only five years ago.
Connecting to 20+ exchanges scattered across the globe is a real technical challenge.
To address this, we have built a custom aggregation micro-service that handles data stream management while guaranteeing the highest level of SLA.
The volatile nature of digital assets leads to fast-changing regimes of data quantity to absorb. Thanks to the combination of micro-services and container-orchestration we can easily absorb peak demands by autoscaling the needed resources.
Because there is no standard in crypto, every Byte we aggregate has to be post-treated and is enriched to match our internal data standard.
Algorithmic trading remains about speed before anything else. To this extent, processed data needs to be made available to consumers in a matter of microseconds.
To facilitate and accelerate micro services communication, we use the latest and fastest PubSub protocols combined with serialized structured data.
To make these dataset accessible to any services instantly, we rely on in-memory databases with replicas that ensure high level of availability.
Storing these large amount of data would have been very complex in the past, notably because of IOPS limitations and limited size scalability.
Thanks to clustered cloud databases, we are able to ensure long term persistent storage at low cost.
Because the data has been completely harmonized, performing analysis is made 10x easier and we don't have to worry about the underlying exchange, feed or asset structure.
To achieve the best performances, we have optimized every step of the process and are dynamically assigning dedicated resources to the streams that need it the most.
To lower exchange-to-infrastructure latency, we are using a simple but efficient private backbone and collocation with major exchanges when available.
All in, this translates into a tick-to-order below 5ms (P95).
One of the primary concern was also to be able to easily scale, both from a technical and operational perspective.
Eventually, most of our tech stack was chosen with industry-reknown technologies to guarantee continued support and be able to onboard new engineers easily. Keeping in mind that any dev stack should be set to evolve, we have strong culture of technological intelligence to identify the next opportunities.
If you are interested to work with us, have the expertise and curiosity in one of the technology we use, feel free to reach out for business and job opportunities.
Connect with us and let's see how Flowdesk can help you navigate the digital asset markets.