Digital^Shift - Amelia.2

Conceptualization and development of a high performance genAI backend

Published on Oct 09, 2024

Reading time: 3 minutes.

Built with

Rust
Tokio
PostgreSQL
Docker

Digital^Shift is a collective of seasoned experts working together to identify and realize opportunities to gain insight, improve productivity, and drive measurable growth.

Synopsis

Our client, a leading innovator in aviation recruitment and training, is revolutionizing how the global aviation industry meets evolving demands.

Digital^Shift partnered with our client to build a solution leveraging advanced AI technologies to streamline recruitment and training processes for airlines and flight schools. Focusing on Evidence-Based Training (EBT) and Competency-Based Training and Assessment (CBTA) methods, it introduces efficient, data-driven approaches to pilot training.

The platform supports complex training requirements by generating personalized Evidence-Based Training reports, tailored to each pilot’s unique strengths and competencies. These customized reports drive targeted interventions to improve training outcomes, all while adhering to EBT and CBTA principles. Built on a scalable AI platform, powered by Dataiku and hosted on AWS, it processes pilot training data from multiple sources. Leveraging Generative AI and machine learning, it provides real-time KPI monitoring, instant feedback, and customized reports, significantly improving the efficiency and flexibility of pilot training programs for both airlines and educational institutions.

My work

I had multiple roles and worked across various fields of work in this project due to the tight timeline of only 3 months from contract start to MVP. The first step consisted of adapting the client’s expectations and requirements to a proper technical specification for implementation. I worked in a team, with my focus being the backend server infrastructure and general aspects of hosting the product. I worked tightly with the person working on the frontend specification to deliver a high quality RESTful API and a Websocket-based API for real time updates. Furthermore I was crucial in specifying the API between our product and the used AI platform (Dataiku).

After the design phase finished, it was my task to set up a proper DevOps pipeline featuring Forgejo as our Git server and Woodpecker CI as our CI server, a CI agent for running the build pipelines was deployed on my server to facilitate the power of concurrency to run build jobs fast and effective. For the sake of simplicity for the others on the project, I set up OAuth on the Git server facilitating Microsoft’s Exchange login with the email addresses provided by Digital^Shift. I designed the CI pipelines to automatically build docker images of the backend and frontend servers and pushing those to Amazon’s AWS ECR, from where ECS pulled the images for deployment.

After this initial setup phase, I kept monitoring, updating and tweaking the Git and CI setup. However my main effort was the development of the Amelia.2 backend server. I decided to use Rust as language for this task, facilitating a tech stack consisting of the Tokio runtime, Axum as web framework and SQLx for interacting with the Postgres database. The main scope of tasks for the backend was access management and generation of EBTs and invoices. For implementation I facilitated a Model/View/Controller (MVC) architecture to keep the application logic separated from infrastructure code (like serializing and deserializing API request or database access) and from the data models themselves, allowing each component to be tested and reused independently from the others.

Backend features

  • RESTful API
  • Authentication with username/email, password and OTP (App TOTP and Email TOTP)
  • User sponsorship flow (users get created by their superior, then set their own password with a mailed login token)
  • Authorization based on user roles
  • Websocket API for live updates
  • Invoice generation based on active users
  • Email service using Mailgun and HTML templates utilizing askama to send invoices, login emails
  • Audit logs for all actions
  • Dynamically generated scope- and time-limited access tokens for callbacks from Dataiku
  • Request validation based on predefined rules, using the validator crate