Datasteroid

Role

Fullstack Engineer (Frontend heavy)

Description

Datasteroid is a low code/no code data orchestration and automation platform that helps automate data transformation workflows ready to be used by downstream consumers. It is an advanced data management platform that consolidates and streamlines the data sources into one unified system. It enhances the ability to manage, analyze, and utilize data, transforming raw data into actionable insights for informed decision-making. It automates complex, resource-intensive data pipelining and deliver structured data compatible for analytics and AI-integrated solutions.

Tech Stack

• Frontend: Typescript, ReactJS, MUI, Socket IO Client, Redux, Konva JS, Apex Charts.

• Backend: Node JS, Fastify JS, Python, Rabbit MQ, PySpark, Polars, Socket IO Server.

• DevOps: Docker, Kubernetes.

• Database: PostgreSQL.

Responsibilities

• Design and implement global layout components.

• Collaborate with the UI team to finalize design feasibility and functionalities.

• Improve the authentication mechanism for the backend with support for third party logins.

• Improve socket IO connection retention.

• Implement a new solution for the data pipeline component to mitigate state synchronization and scalability issues.

• Develop a data visualization component module having the ability to display multiple charts and changing their size and location on the viewport.

• Implement an in app notification system..

Impact

• Designed scalable global layout components leveraging component composition and Context API to ensure efficient state management and seamless data propagation to child components. This approach significantly improved maintainability and scalability across the application.

• Collaborated with the UI team to finalize designs ahead of deadlines, ensuring feasibility checks for both design and functionality. This proactive effort enabled an early start on component architecture design, accelerating overall development timelines.

• Implemented a secure token-based authentication system with refresh token renewal mechanisms and secure cookies, enabling seamless integration of third-party logins and enhancing overall security.

• Increased Socket.IO connection retention from 80% to 99% by designing a singleton socket management class. The use of user IDs as room identifiers ensured reliable and efficient real-time communication.