Software Engineer, Data Infrastructure
About the company -----------------
Companies want to train their own large models on their own data. The current industry standard is to train on a random sample of your data, which is inefficient at best and actively harmful to model quality at worst. There is compelling research showing that smarter data selection can train better models faster—we know because we did much of this research. Given the high costs of training, this presents a huge market opportunity. We founded DatologyAI to translate this research into tools that enable enterprise customers to identify the right data on which to train, resulting in better models for cheaper.
Our team has pioneered deep learning data research, built startups, and created tools for enterprise ML.
We have raised over $11.5M from top-tier investors including Amplify Partners, Radical Ventures, Conviction Capital, Jeff Dean, Yann LeCun, Geoff Hinton, and Adam D’Angelo to help make our vision a reality. Learn more about the company here.
This role is based in Redwood City, CA. We are in person 5 days per week and offer relocation assistance to new employees. We provide visa sponsorship for candidates selected for this role.
About the role --------------
We're looking for an experienced Data Platform Engineer to join as a member of our core Datology AI team. As one of our early senior hires, you will partner closely with our founders on the direction of our product and drive business-critical technical decisions. You will lead the development of our core product and data platform. These are key components of our stack that allow us to process customer data and apply state of the art research for identifying the most informative data points in large-scale datasets. You will have a broad impact over the technology, product, and our company's culture. We provide visa sponsorship for candidates selected for this role.
As a Data Infrastructure Engineer at Datology AI, you will be responsible for:
- Design, build and maintain highly scalable data processing solutions, while ensuring scalability, reliability, and security
- Architect, build, and deploy the back-end systems and services that power our data curation platform
- Partner with researchers and engineers to bring new features and research capabilities to our customers
- Ensure that our systems are reliable, secure, and worthy of our customers' trust
About You ---------
There are a few specific things we’ll be looking for that will help you succeed in this role:
- Have meaningful experience with leading and building production data systems to deliver on major product initiatives.
- You have built and managed highly scalable data processing solutions (e.g. Spark, Flink), data lakes or warehouses (e.g. Snowflake, Hive), authored queries (SQL), distributed storage systems (e.g., HDFS, S3), used workflow management (e.g. Airflow, Dagster), and have experience maintaining the infra that supports these.
- Proficiency in at least one programming language commonly used within Data Engineering, such as Python, Scala, or Java.
- Expertise with any of ETL schedulers such as Airflow, Dagster, or similar frameworks.
- Experience maintaining a high quality bar for design, correctness, and testing.
- Take pride in building and operating scalable, reliable, secure systems
- Have a humble attitude, an eagerness to help your colleagues, and a desire to do whatever it takes to make the team succeed
- Own problems end-to-end, and are willing to pick up whatever knowledge you're missing to get the job done
We would love it if candidates have:
- You've led or managed a Data Engineering / Platform / Infrastructure Team.
- Experience building ML/DL systems and/or data infrastructure that feeds into training large ML models
Compensation and Benefits -----------------------------
At DatologyAI, we are dedicated to ensuring pay fairness and maintaining highly competitive market rates. We also offer a comprehensive benefits package to support our employees' well-being and professional growth:
- Comprehensive health, dental, and vision insurance for you and your dependents.
- 401(k) plan with a generous 4% company match.
- Unlimited paid time off (PTO) policy – we encourage employees to take a minimum of 2 weeks per year to rest and recharge.
- Annual $2,000 wellness stipend to support your physical and mental well-being.
- Annual $1,000 learning and development stipend to foster continuous growth and skill acquisition.
- Daily lunches and snacks are provided in our office!
- Relocation assistance for employees moving to the Bay Area to help ease the transition
We believe in investing in our employees and providing them with the resources they need to thrive both personally and professionally.
About DatologyAI
DatologyAI optimize training efficiency, maximize performance and reduce compute costs.
More jobs at DatologyAI
- AI Developer Experience & Media Lead
- Research Engineer
- Research Scientist
- Product Manager
- Software Engineer, Front-end
- Software Engineer, Cloud Infrastructure
- Solutions Engineer (AI/ML, Pre-Sales)
- Research Scientist, Post-Training
- Software Engineer, Infrastructure