Transitioning from Academia to Data Science - Jake Klamka with Kevin Hale

Insight Data Science offers free fellowships to help scientists and engineers transition to careers in data science and AI. The program is full-time and funded by companies, with fellows building projects for seven weeks and meeting top data teams for potential hiring opportunities. The struggles faced by mathematicians, neuroscientists, biologists, and engineers in entering data science or AI fields are highlighted. Companies want data scientists who have technical skills such as SQL and Python, but they are particularly impressed by candidates who have a side project showcasing their passion. There are three main types of data science roles: product analytics or business analytics roles, data product roles, and AI or machine learning engineering roles. Common pitfalls for people transitioning into data science include neglecting the importance of communication and understanding the business and product problems. Insight, a data science program, has scaled up and expanded its curriculum to include specializations in data science, data engineering, health data, AI, and product management. The program involves fellows transitioning from academia to data science, working on real-world problems and learning quickly. Good projects on a data science resume should be product-focused and provide real solutions. More data scientists are likely to become founders in the future. Teaching product in the context of transitioning from academia to data science emphasizes learning by doing and iterating. Data cleaning is an essential task in data science, and startups should consider preventing data cleaning issues. Tracking and optimizing specific metrics is crucial for success in a business or startup. Understanding user behavior and predicting churn is important for intervention strategies. Data science does not have an ideal background, as successful data scientists come from various fields. Insight is a program where fellows work with startup companies on projects, with a main focus on companies hiring the fellows. Contracting in data science helps startups in the prototyping stage. Jake Klamka is excited about the field of health in data science, particularly its potential impact on healthcare.

Kevin's intro

Kevin Hale, a partner at Y Combinator, founded wufu ring and was acquired by SurveyMonkey. He has been with Y Combinator since then.

Jake's intro

Applying to YC with one product then changing it

The transition from academia to data science and the challenges faced by individuals with technical backgrounds who want to enter the field are discussed. The importance of starting with a problem and finding a solution is emphasized. The struggles faced by mathematicians, neuroscientists, biologists, and engineers in entering data science or AI fields are highlighted. The need for a solution to help these individuals overcome barriers and successfully transition into the industry is emphasized. The process of applying to Y Combinator with one product and then changing it is also discussed, with the speaker realizing the suitability of an in-person program and teaching a class to facilitate learning.

How Insight started

Insight is a program that aims to bridge the gap between academia and industry.
It offers a one to two month program for data scientists to learn skills and techniques.
The goal is to make these data scientists immediately valuable to companies.

Jake's first students and initial coursework

The most profound aspect of Jake's first students and initial coursework is the success of the program in placing all eight fellows in top data science positions.

Jake started his first group of students for his data science program in 2012.
He confirmed the interest of his friends in academia and the demand from hiring managers for individuals with data science skills.
The program focused on bringing in leading data scientists from companies like Facebook, LinkedIn, Twitter, and Square to teach the fellows.
The first class had eight fellows, all of whom were successfully placed in top data science positions.
Despite lacking a track record, Jake attracted applicants by promising access to industry professionals.
He sought students who were genuinely excited about making an applied impact in the world.

Finding out what companies want from data scientists

Companies want data scientists who have technical skills such as SQL and Python, but they are particularly impressed by candidates who have a side project showcasing their passion. Curiosity is valued, as it demonstrates a willingness to explore and try new things. Diverse backgrounds and interesting projects outside of academia are highly sought after. The video explores the transition from academia to data science and what companies expect from data scientists.

Picking the first class of students

The first class of students for a data science program was selected based on passion and demonstrated skills in data science. The selection process involved conducting interviews and considering candidates from diverse backgrounds. A mathematician who had done impressive data analysis projects on the side was chosen, showcasing their ability to quickly learn and apply new skills. This individual went on to have a successful career at Facebook.

Common pitfalls for people transitioning into data science

Common pitfalls for people transitioning into data science:

Individuals with a high level of technical knowledge often focus too much on algorithms and technical skills, neglecting the importance of communication and understanding the business and product problems.
Data scientists should think about the goals of the company and how their skills can be used to achieve those goals, rather than solely focusing on their own technical abilities.
It is important to understand the problem at hand and figure out how to fit into the team or organization, rather than simply showcasing one's own skills.
Understanding the mission and goals of the company is crucial, as well as positioning oneself as a solution rather than just having a bunch of skills.
The definition of data science is broad and should be defined in the context of the specific job or company.

Types of data science roles

There are three main types of data science roles: product analytics or business analytics roles, data product roles, and AI or machine learning engineering roles.

Product analytics or business analytics roles involve analyzing data to understand and improve user and company performance.
Data product roles use machine learning and predictive models to enhance the user experience and provide desired features.
AI or machine learning engineering roles focus on building products that are entirely based on machine learning and their success is dependent on it.

What data scientists should look out for in companies

Data scientists transitioning from academia to industry should consider the following when evaluating companies:

Both the data scientist and the company should understand the actual problem that needs to be solved.
Beware of vague requests for deep learning without a clear understanding of its purpose.
Look for companies with a clear mission and alignment where your skills can contribute to their success.

Chuck Grimmett asks - When do you know you need to bring in seasoned data scientists?

The most profound aspect of the topic is determining when to bring in seasoned data scientists in a startup.

Key points:

Determine if data is critical to the product or if it can be added later for optimization.
If data is critical from day one, hire a data scientist or machine learning engineer early on to set up the infrastructure for success.
Seek advice from industry professionals or consider hiring a data science advisor if not ready to hire a data scientist yet.

How Insight has scaled and changed

Insight, a data science program, has scaled up and expanded its curriculum to include specializations in data science, data engineering, health data, AI, and product management. The program is divided into different classes depending on the specialization and lasts for seven weeks. The demand for specialized roles in data science has increased, with the need for data engineers, data scientists, and machine learning engineers. Each program has a small number of fellows to encourage collaboration and project work.

What happens in the program

The program involves fellows transitioning from academia to data science. They have the option to come up with their own project or partner with a YC startup. The program is collaborative, resembling a startup office, with fellows working together and helping each other. The goal is to execute real-world problems and learn quickly.

Fellows transition from academia to data science
Option to come up with own project or partner with a YC startup
Projects need to be built quickly and presented to the team
Collaborative environment resembling a startup office
Fellows work together and help each other
Focus on executing real-world problems and learning quickly

Examples of a good project for a data science resume

The most profound aspect of the topic is the importance of having good projects on a data science resume that are useful, actionable, and provide tangible value.

Key points:

Projects should be product-focused and provide real solutions
Avoid showcasing only technical skills or generic analysis
Examples of a bad project include analyzing general trends without a clear call to action
Examples of a good project include building a predictive model to help homeowners determine the profitability of buying solar panels for their specific location
Good projects increase the chances of getting hired in the field of data science.

Will more data scientists be founders in the future?

More data scientists are likely to become founders in the future.

Approximately a quarter of participants in the fellows program express interest in starting their own company within the next five years.
Some alumni have already started their own companies, such as Trace Genomics and Spring Discovery.
Trace Genomics uses genomic data for agricultural purposes, while Spring Discovery focuses on aging-related diseases using machine learning.
The founder spirit and skill set are present in data scientists.
Understanding product is crucial for success as both an employee and a founder.

Teaching product

Teaching product in the context of transitioning from academia to data science emphasizes learning by doing and iterating, rather than just theory. The learning experience involves building a product and receiving continual feedback to improve. Working with dirty data is a challenge in data science.

Cleaning data

Data cleaning is an essential and time-consuming task in data science. It involves determining relevant data, combining datasets, and ensuring clean and meaningful data. Startups should consider preventing data cleaning issues. Founders often overlook important details like user logins, rendering data unusable. The video emphasizes starting to track data early on, even simple logging, to build a foundation for analysis and prevent regrets. Involving a data scientist is essential for tracking data effectively.

Tools for tracking data

Tools for tracking data in the transition from academia to data science often require teams to create their own tools due to a lack of sophisticated options. It is important to carefully consider what data to track based on the specific goals and needs of the product.

Track what are you trying to optimize

The most profound aspect of the topic is the importance of tracking and optimizing specific metrics in order to achieve success in a business or startup.

Key points:

Identify one or two key metrics that align with the company's goals and focus on optimizing them.
Examples of metrics tracked by companies like Netflix and Khan Academy include user engagement and learning time.
The overall goal for most companies is growth, and it is crucial to track and optimize the key performance indicators (KPIs) that drive revenue.
For most companies, the main focus is on conversion and churn for revenue, as well as engagement.
Questions can be categorized into increasing conversion or reducing churn for revenue, and the same for engagement.

Churn and conversion

Churn and conversion are important aspects in the transition from academia to data science. Understanding user behavior and predicting churn is crucial for intervention strategies. Improving conversion rates and prioritizing churn can help companies identify what is working or not working for the user. Exploratory data analysis, A/B testing, and multi-armed bandit testing are key tools for understanding user behavior. Metrics such as cohorts and retention curves are important for improving churn. Data science plays a crucial role in addressing churn and improving conversion rates. Open-source tools like Python and Jupyter notebooks are commonly used in data science teams for building deep models. Data scientists work directly on the product, similar to an engineering team.

Is there an ideal background for a data scientist?

Data science does not have an ideal background, as successful data scientists come from various fields such as physics, engineering, psychology, and archaeology. Different backgrounds bring different perspectives and skills to the field. Collaboration between individuals from diverse backgrounds is highly valued in the data science field.

Successful data scientists come from various fields such as physics, engineering, psychology, and archaeology.
Different backgrounds bring different perspectives and skills to the field.
Mathematicians make great data engineers because they think about large-scale systems.
Social scientists excel at understanding people and writing effective questions.
Collaboration between individuals from diverse backgrounds is highly valued in the data science field.

Which startups recruit well at Insight?

Insight is a program where fellows work with startup companies on projects, with a main focus on companies hiring the fellows.
Startups that recruit data scientists well emphasize the impact the data scientists will have on the company's success.
The impact is the biggest factor in attracting data scientists, even more important than the technical aspect.

Contracting

Contracting in the context of data science is beneficial for startups in the prototyping stage, allowing them to work with consultants or fellows to test and improve algorithms. It is useful for determining feasibility and improving prototypes before integrating them into the product. In the context of transitioning from academia to data science, contracting requires adaptability and evolution in the field, as relying solely on overseas contract software engineers is not feasible due to constant product changes.

Contracting in data science helps startups in the prototyping stage
Consultants or fellows can be hired to test and improve algorithms
Feasibility and prototype improvement are key benefits of contracting
Contracting is important before integrating prototypes into the product
Transitioning from academia to data science requires adaptability and evolution
Relying solely on overseas contract software engineers is not feasible due to constant product changes.

Fields Jake is excited about

Jake Klamka is excited about the field of health in data science, particularly its potential impact on healthcare. He believes that data science and machine learning can contribute to early disease detection and monitoring. For instance, at Memorial Sloan Kettering Cancer Hospital, data scientists and engineers develop data products to assist doctors in recommending personalized clinical trials, potentially saving lives. Jake is also enthusiastic about the broader application of data science beyond business optimization, as more companies recognize its value.

Key points:

Data science and machine learning can significantly impact healthcare.
Early disease detection and monitoring are areas where data science can contribute.
Memorial Sloan Kettering Cancer Hospital uses data products to help doctors recommend personalized clinical trials.
Data science has the potential to save lives in healthcare.
Data science is not limited to business optimization and has broader applications.