AWS Consultants for Data Lakes & Data Engineering
AWS consultants are specialists who plan, configure and run workloads on Amazon Web Services. Scorchsoft is a Birmingham-based application developer: we build the apps, portals and dashboards that sit on top of your data lake, and we configure the AWS services they depend on.
Scorchsoft sits at the application layer. Most of our work is the software people actually use — portals, internal tools, dashboards, integrations and AI features — plus the AWS configuration those applications need to run: S3 storage, PostgreSQL databases, containers, access policies and deployment pipelines. This website is one example. It runs in Docker containers on AWS, stores its media in S3, keeps its content in PostgreSQL and deploys through GitHub Actions.
That is where we focus, and we go deep there. We work on data projects of most shapes, often alongside an in-house data team or the supplier who runs the platform underneath. If a brief reaches past what we do well, we will say so early and can recommend options or make an introduction. For the platform side of a lake, read our data lake development page; for hosting and scaling applications on AWS, read AWS app development. If you are still deciding whether a lake is the right structure at all, our guide on when your business needs a data lake works through that choice.
What our AWS consultants actually do
The three things clients most often ask us for when data sits at the centre of a project.
The applications on top of the data
A data lake earns its keep when somebody can act on what is in it. That is the part we build: the portal a supplier logs into, the dashboard a manager reads on a Monday morning, the approval workflow, the export finance needs at month end, the AI feature that answers questions over your own records. We have written that kind of software since 2010. We treat your platform as the source rather than something to rebuild, whether it is AWS, Microsoft Fabric, a SQL Server or a warehouse another supplier runs. See our work on data dashboards and portal and SaaS web apps.
The AWS configuration those applications need
Applications do not run on their own, so we configure the AWS services around them: S3 for file and media storage, RDS or Aurora for the database, containers for the app itself, IAM roles and policies, CloudFront, backups, monitoring and a deployment pipeline. Ingestion is often application code too — an API pulled on a schedule, a nightly export landed and curated — and we write it. This is infrastructure work in service of an application, not a separate platform practice. Related reading: AWS app development and SQL database development.
Lake On Rails, our own data product
Lake On Rails is our data governance software for businesses of 50 to 1,000 people. It sits above Microsoft Fabric or AWS and never moves or reads the contents of your data. It holds a register of every dataset and report with a named owner, trust tiers, 7 written procedures, approval workflows and an audit trail, then scores whether that is still holding a year on. Plans start at £8,000 a year. We built it because the same question came up at the handover of every data platform we delivered: who owns this number? Read more about Lake On Rails.
Where we focus
Most data projects have an application at the sharp end: the thing somebody logs into, reads or acts on. That is where our work concentrates, and the AWS configuration we do sits in service of it. The rest of your data landscape we are happy to work around, or alongside.
What we build
- Applications, portals and dashboards built on top of your data
- AWS configuration for those applications: S3, RDS, containers, IAM, CI/CD
- Ingestion by API, scheduled export or integration, written as application code
- Data governance through our own product, Lake On Rails
- Integrations between your apps, your lake and third-party platforms
How we fit with the rest of your data landscape
- Building on top of Databricks, Snowflake, Microsoft Fabric or a warehouse another supplier runs
- Working alongside your in-house data team or your existing platform partner
- Picking up the application, integration and reporting half of a wider data programme
- Bringing in a specialist where a brief calls for one, or making the introduction ourselves
- Giving you a straight read on scope in the first conversation, before anyone commits
Not sure where your project sits? Tell us what you have and we will give you a straight read.
Talk to a UK AWS consultant
Tell us what your data platform looks like today and what you want people to be able to do with it. We will come back with our thoughts and a rough cost estimate.
AWS and data engineering questions
An AWS data lake is a central store, usually built on Amazon S3, that holds raw and curated data from many sources in one place without forcing a reporting model on day one. Around it sit services that make the files queryable and governable: the Glue Data Catalog for table metadata, Athena for SQL over files, and IAM for access control. A lake is not a warehouse. A warehouse holds curated, structured data tuned for fast reporting; a lake keeps the raw history flexible so you can build several views of it later. AWS sets out the pattern at aws.amazon.com, and our data lake development page covers the platform side in more detail.
Scorchsoft is a UK application development company, and the application layer is where we concentrate: the software that uses your data, and the AWS services that software needs. We work on data projects of most shapes, frequently alongside an in-house data team or the supplier who runs the platform underneath. If a brief needs expertise outside our usual remit, we will say so early and can recommend options or make an introduction.
Yes, at the application layer. That means ingestion written as application code, API and file integrations, transformation into the shape an app or dashboard needs, scheduled jobs, and the AWS plumbing around them. For most mid-sized organisations that is the bulk of what a data platform is asked to do. If your project calls for something more specialised, tell us what you have and we will give you a straight view on scope.
Yes. The application layer is the same work whichever platform holds the data, and our own product, Lake On Rails, is built to sit above either Microsoft Fabric or AWS. We have also built on SQL Server and PostgreSQL where that is what a client already runs. See our SQL database development page.
This page is about data: lakes, ingestion, governance and the applications that read from them, plus the AWS consultancy around that. Our AWS app development page is about hosting and scaling applications on AWS generally, using services such as EC2, RDS, DynamoDB, S3 and Lambda. If your question is "can you host and scale my app", that page is the better read.
We work in-house from Birmingham, in the Jewellery Quarter, and have been building software for other people's businesses since 2010. The people who scope your project are the people who build it. You can see systems we have delivered in our portfolio.
Experience Delivering
Capabilities we regularly build alongside data work, from cloud databases and dashboards to integrations and automation. The full list is on our capabilities page.
Discuss Your AWS or Data Project
Tell us where your data lives today and what you want people to do with it. We will respond with our thoughts and a rough cost estimate. Scorchsoft is a UK team of web and mobile app developers and designers. We operate in-house from Birmingham, and our offices are in the heart of the Jewellery Quarter.
Building for teams across healthcare, logistics, motorsport and manufacturing
What Our Clients Say
Scorchsoft helped us take our idea for an app and make it a reality. Everything from the planning meeting to decide what we really needed to the project management and execution was great. It was delivered on time - early in fact - and on budget. Highly recommend.
Rebecca PalserDragonfly IntelligenceScorchsoft is a brilliant company with fantastic knowledge of the mobile app industry. From the project management to the development team, they have been the perfect candidate for our project, and we can't thank them enough!
Lance ChorltonGapped OnlineWe're really pleased with the work Scorchsoft has done in developing our web portal! They have been accurate with timelines and budget, delivering a solid product that allows us to monitor and manage patients remotely while they use our novel medical device at-home. The "plan - design - build" approach has worked well and saved us time in the long-run by catching requirements and issues early.
Daniel GreenSensTrainI'm really pleased with how my app came out, it was exactly what I was looking for. The team at Scorchsoft are great at what they do and made the whole process as simple and easy as possible. Being someone who is not very tech savvy the set up and back end operations were done in a great easy to use manner even for myself which makes using my app stress free. Thanks to all the team!
Ruben CarrollMosaic MasterpiecesThe new Flourish Education website has already removed a lot of manual processes, freeing up both schools, candidates and internal employees time. We are delighted with the look and feel which is clean, professional and more engaging. We are also pleased with the decision to have an HD video background on the homepage, and building immediate trust with our clients by giving them a taste of what it looks like in the Flourish Education offices.
James HancocksMarketing Manager, Flourish Education I can't believe how quickly we started to see results with this project. Scorchsoft provided us with graphic-designed mockups of how the app would look once built, and we were able to sell the product for use by our first customer before the product was finished. Since launching in March, we have secured a major television network as a client who now uses Image Approvals to manage the talent approval process for their productions.
Aimee SpinksMD, ImageApprovals.com
Need help building your ideas?
Tell us where you're headed and we'll come back with our thoughts, a realistic plan and a rough cost estimate. Scorchsoft is a UK-based team of app, portal and AI developers, working in-house from Birmingham's Jewellery Quarter.