Data Council 2024 Recap
A Dive into Data Reliability Engineering and More
I recently had the privilege of speaking at Data Council 2024, held in the vibrant city of Austin. As a data reliability-focused engineer, I was eager to share insights and experiences from my approach to Data systems with a broader facet of the Data Community. I wanted to share my experiences at the conference and what I learned from the community there.
Data Reliability
My talk, Scaling Data Reliability, centered around introducing Data Reliability Engineering (DRE) as a crucial practice for growing data-intensive tech stacks. I emphasized the value of DRE in enhancing data quality, reliability, and overall operational efficiency. I enjoy discussing DRE because it helps me refocus on a stakeholder-first approach to data engineering.
In my talk (recording coming soon), I highlighted how DRE helps data teams build robust infrastructures by managing expectations for uptime and data quality. In a conference predominantly focused on showcasing tools and products, the concept of DRE sparked numerous conversations. Its methodology, resonated well, driving discussions on implementation strategies. The major follow-up interest from data teams underscored the need for DRE in today's data landscape and also the growing importance of data for the company. The question was,
“How do we get buy-in from our company to hire a data reliability engineer?”
I am still trying to figure out the answer, but if any of you figure out how to capitalize on generative AI hype to emphasize the importance of a stable data stack LMK.
Summary of Data Council
Beyond my talk, Data Council 2024 was a great generalized Data conference (and affordable). With three days of three tracks, the conference provided tons of information, too much for me. I stuck to data engineering and Mlops since those are the local Meetups run. There was an interesting emphasis on venture capitalist seed funding. During one of the conference happy hours there was a VC pitch meeting. I am glad for the emphasis on growth during an interesting economic climate, but personally, I am taking a friend's advice and doubling down on open-source and community-driven data products.
Post-Conference Socials
One of the standout features of the Data Council was its robust post-conference social events. Each evening, there were various events (usually vendor-sponsored) at different venues across Austin. As someone traveling from out of town, these socials were refreshing. I didn’t have to think about our plan for my evening activities. this also made networking easier, which is the true purpose of attending every conference.
If you're looking for a fun conference next year, I recommend Data Council. It's easy to travel to. There’s a variety of topics good for an entire date team. I enjoyed my experience and I hope others did as well.
