Gurjeet Singh: Shaping the Future of Data [Entire Talk]

TL;DR

Ayasdi seeks to turn large datasets into knowledge automatically by combining results from hundreds of machine-learning algorithms before an analyst enters the process. Its approach grew from Stanford research in algebraic topology and a DARPA-backed project, then became a company after its founders explored customer needs and identified applications for the technology.

Transcript

It's a real pleasure to be introducing Ayasdi here and the co-founders. Just briefly, I'm going to give you the titles of the gentlemen sitting to my right here. This is Gunnar Carlsson, who is also a professor here at Stanford. He holds the Anne and Bill Swindell Professorship in the school of Humanities and Sciences. And this is also Gurjeet Sing... Read More

Key Insights

• Ayasdi is designed to turn data into knowledge by automating much of the analytical process and presenting potential answers before a person becomes involved. Its stated business objective is to perform this transformation in close to zero time, rather than beginning with manually formulated questions.
• The conventional analytics workflow is driven by hypotheses created by skilled analysts. An analyst translates an idea into a query or code, runs it against a database, and evaluates the result, which may or may not confirm the original idea.
• Data-science work requires a difficult combination of mathematics, statistics, computer science, and domain knowledge. Gurjeet Singh argues that this need for advanced and specialized expertise creates a people problem for organizations attempting to derive knowledge from large datasets.
• The number of possible hypotheses in a table is exponential in the table's dimensions and size. This makes manual exploration fundamentally constrained because analysts cannot realistically formulate and test every potentially meaningful question contained within a large dataset.
• Ayasdi's methodology processes large amounts of data through hundreds of machine-learning algorithms. The system then combines their results using research developed at Stanford, allowing users to begin their work with possible answers instead of starting from an entirely blank hypothesis.
• Algebraic topology provided the mathematical foundation for the research behind Ayasdi. Gunnar Carlsson began exploring its possible value for understanding datasets in the mid-1990s, secured initial support for a half-year postdoctoral position, and later expanded the effort into a larger DARPA project.
• Ayasdi grew from a university research project into a commercial company through technical development and customer discovery. An early investor first encountered the opportunity through four mathematical papers, then worked with Gurjeet Singh as he investigated customer needs before Singh accepted funding.
• Gurjeet Singh's Stanford path was shaped by persistence and practical problem solving. After arriving with funding for roughly one quarter, he contacted professors across Stanford, emphasized his programming ability, and proposed computational approaches in fields he initially did not know, including computational fluid dynamics and computational mechanics.

Questions & Answers

Q: How does Ayasdi turn large datasets into knowledge?

Ayasdi processes large amounts of data through hundreds of machine-learning algorithms and combines the resulting outputs using research developed at Stanford. Instead of requiring an analyst to begin by inventing a hypothesis, writing a query, and testing it against a database, the system aims to generate potential answers before a person first enters the analytical process.

Q: Why is hypothesis-driven data analysis difficult to scale?

Hypothesis-driven analysis depends on a skilled person forming an idea, translating it into code or a query, running it against a database, and interpreting the result. The process becomes difficult to scale because a table can contain exponentially many possible hypotheses based on its dimensions and size, while analysts can examine only a limited selection of them.

Q: What skills does a data scientist need according to Gurjeet Singh?

A data scientist needs some combination of mathematics, statistics, computer science, and knowledge of the relevant domain. Gurjeet Singh presents this breadth as a significant staffing problem because the hypotheses involved can be complex and the necessary expertise is specialized. He also notes that becoming a data scientist generally requires advanced education rather than a short or informal path.

Q: How did algebraic topology contribute to Ayasdi?

Algebraic topology supplied the abstract mathematical ideas that Gunnar Carlsson sought to apply to understanding datasets. He began considering these applications in the mid-1990s after spending most of his career in pure mathematics. Initial funding supported one postdoctoral researcher for half a year, and the work later expanded through additional funding into a larger DARPA project.

Q: How did Ayasdi emerge from Stanford research?

Ayasdi emerged from a Stanford research effort that investigated how topology could help analyze datasets. Gunnar Carlsson developed the mathematical direction, and Gurjeet Singh worked with him during the larger DARPA project that followed the initial research. The commercial spin-off came from that project, translating technical insights developed inside the university into the basis of a company.

Q: Why did Gurjeet Singh initially reject a $1 million investment?

Gurjeet Singh initially declined the $1 million check because he did not yet know what he would do with the money. Rather than accepting funding immediately, he continued customer-development work and reported what he learned to the prospective investor. After they worked together for a period and the business opportunity became clearer, he eventually agreed to accept the investment.

Q: How did Gurjeet Singh finance his early Stanford studies?

Gurjeet Singh arrived at Stanford with roughly one quarter of funding assembled by his family. To find support, he wrote a program that collected Stanford professors' email addresses and contacted them about his programming abilities and need for work. His outreach led to meetings, including one with Anthony Jameson in the Aero Astro Department, who agreed to work with him.

Q: What entrepreneurial lessons come from Gurjeet Singh's research path?

Gurjeet Singh's path shows the value of approaching unfamiliar technical problems through capabilities that can create immediate value. Although he did not know computational fluid dynamics when meeting Anthony Jameson, he recognized its need for computational scale and proposed using clusters of digital signal processors. He later built prototypes, presented work to Boeing Phantom Works, approached the NSF, and repeatedly sought new research support.

Summary

This video features an interview with the co-founders of Ayasdi, Gunnar Carlsson and Gurjeet Singh. They discuss the journey of turning mathematical research into a successful company that specializes in turning data into knowledge using automated methodologies. They explain how topology, the study of shapes, plays a significant role in their technology and how they have applied it to various industries such as pharmaceuticals, finance, and healthcare.

Questions & Answers

Q: What problem is Ayasdi trying to solve?

Ayasdi aims to solve the problem of turning data into knowledge using automated methodologies.

Q: What is the standard process for turning data into knowledge?

The standard process involves a smart analyst coming up with a hypothesis or idea, converting it into a query or code, running it against a database, and analyzing the results.

Q: What are the problems with the standard process?

The first problem is the need for specialized people with advanced degrees in mathematics, statistics, computer science, and domain knowledge. The second problem is that there are too many hypotheses to consider, especially with large datasets.

Q: How does Ayasdi's methodology differ from the standard process?

Ayasdi uses a more automated methodology by applying hundreds of machine learning algorithms to large datasets. Their algorithms combine the results based on research conducted at Stanford, allowing users to have some answers from the start.

Q: What is Gunnar Carlsson's background and how did he start working on applied mathematics?

Gunnar is a mathematician who had a background in algebraic topology. He started exploring the application of mathematical concepts to real-world problems in the mid '90s.

Q: How did Gurjeet Singh end up in the Ph.D. program at Stanford?

Gurjeet, originally from India, had a passion for mathematics and wanted to learn more to expand his career opportunities. He found a program called Scientific Computing at Stanford and applied, hoping to combine his knowledge of mathematics and computer science.

Q: How did Gurjeet and Gunnar start working together?

Gurjeet saw an email from Gunnar, who was talking about using algebraic topology to understand large complex datasets. He saw an opportunity to apply his math and machine learning skills and reached out to Gunnar. He became a student of Gunnar's and started working together on research projects.

Q: What was different about Gurjeet compared to other students?

Gurjeet not only had a deep understanding of the theory but also had the drive to implement the ideas and solve real-world problems. He was able to quickly prototype solutions and show practical applications of the research.

Q: Did Gunnar ever feel like a sellout for starting a company instead of pursuing pure mathematical research?

Gunnar initially anticipated feeling like a sellout, but his colleagues and peers were actually supportive and appreciative of his entrepreneurial endeavors. He believes that applying math to real-world problems is a valuable contribution.

Q: How did Ayasdi transition from theoretical research to building a company?

After completing their research and realizing the potential impact of their findings, Gurjeet and Harlan Sexton left academia and started building Ayasdi. They focused on meeting with potential clients and gaining feedback on use cases that Ayasdi could address. They were able to secure funding after demonstrating the value and impact of their technology.

Q: What are some of the use cases Ayasdi has worked on?

Ayasdi has worked on various use cases, including fraud detection in the banking industry, triage models in healthcare, and analyzing the characteristics of successful doctors in hospitals.

Takeaways

Ayasdi's technology aims to automate the process of turning data into knowledge by applying topology and machine learning algorithms to large datasets. They have successfully addressed problems in multiple industries and have demonstrated significant improvements in data analysis and decision-making. The transition from academia to building a company required a shift in focus and a strong drive to solve real-world problems. Overall, Ayasdi's approach highlights the potential for mathematics and data analysis to revolutionize various industries.

Summary & Key Takeaways

• Ayasdi addresses the difficulty of converting data into knowledge through analyst-generated hypotheses and database queries. That conventional workflow requires specialists with mathematics, statistics, computer science, and domain expertise. It also faces an enormous search problem because the possible hypotheses within even a tabular dataset grow exponentially with the table's dimensions and size.

• The proposed alternative processes large datasets through hundreds of machine-learning algorithms and combines their results using research developed at Stanford. A person first becomes involved after the system has already produced potential answers. Ayasdi therefore aims to automate discovery and reduce the time required to extract useful knowledge from complex data.

• The company emerged from Gunnar Carlsson's effort to apply algebraic topology to understanding datasets and from a larger DARPA project where he worked with Gurjeet Singh. Singh's path included electrical engineering in India, Stanford's Scientific Computing program, computational research, persistent funding searches, prototype development, and customer conversations before accepting an initial investment.

• Key Insights Q&A?