Skip to content

Data Products Ideation#

Disclaimer

This playbook is intended primarily towards product owners, line of business stakeholders and service delivery managers. This playbook assumes that the use cases, and the funding sponsors are identified

Introduction#

The objective of the ideation phase is clarity on the desirability, feasibility and viability of an idea. The result of the ideation is understanding what shape the delivery will have to take: what are all the components that contribute for the idea to be implemented. Whether data engineering, data science, ML, web application development, or standard software have to be employed to learn whether the idea works. It is the first step in the Data & AI delivery model after a demand has been identified.

This guide extends the DP&SE Digital Discovery & Proof of Concept as a service on the data product side. It provides guidance on product (design) thinking, and how to facilitate the conversations.

Every ideation will be different. The participants will have already thought about some of the aspects, some questions will be new to them. The conversations are exploratory, but also align the business and technology sides of the company - all the participants can learn the different constraints from each other.

These conversations are also a good basis to revisit on a regular basis to see what changed in the assumptions and the context. Making sure that the conversation is captured in a visual way, helps new people joining the team to get up to speed with someone who already knows the context. The ideation is not an Excel sheet with questions to be answered in spare time. This will not lead to the desired understanding of the background and the context among all the people who should be involved in the delivery.

Data Products#

"Data products are created specifically for analytical consumption. They have defined and agreed-upon shapes, consumption interfaces, and maintenance and refresh cycles, all of which are documented.

Data products are processed domain data assets or datasets that you can share with downstream processes through interfaces in a service-level objective. Unless otherwise required, you should process, shape, cleanse, aggregate, and normalize your raw data to meet agreed-upon quality standards before you make it available for use."

Quote: Microsoft Learn - What is a data product?

Example: CMC Pre-Clinical Yield

CMC Pre-Clinical Yield of a test production run. This data informs the clinical trial studies about the quantity of output manufactured and is key in designing the studies.

It publishes data on the quantity of output produced, in a format that consumable by the manufacturing execution system (MES), and the operators in their daily tooling (an excel document).

The data product has multiple consumers, primarily in the Chemistry, Manufacturing and Controls (CMC) area. The data is refreshed every time a laboratory result for a batch is entered into the system, with a delay of up to 1 hour. For each completed production batch, the yield result can be delayed by up to 5 working days.

The data covers the laboratory results from the PAS-X instance No. 31 as this is the system for the pre-clinical trial manufacturing.

Data product conversations#

There are three key conversations to consider when ideating a data product:

  • What different data contributes to the overall success of the idea, whether this is analytical or transactional data. A good tool to facilitate this is a data product interaction map.
  • For each of the identified data products, what shape should the data be in. What usage patterns are expected, what promises should be made to the consumers of the data product. Here, a data product canvas can be used.
  • What is the ownership and delivery structure for the products is needed: is it a single team, is it a one-off or a multi-year effort, is it a cross-skill team or a highly specialized one, what is the funding model.

Participants and facilitators#

This playbook provides guidance on how to facilitate the above conversations and what artifacts to produce.

Generally, the facilitators should be comfortable navigating a lot of uncertainty - generalists in technology and business, curious about insights in an intensive process.

In the sessions it is important that the participants consist of:

  • Line of Business representatives to give guidance the initiative and make a call on prioritization and investment commitments
  • Subject Matter Experts to support understanding of which data supports the outcomes of the initiative
  • Technology representatives to synthesize the insights into actionable feasibility questions for exploration

Other stakeholders:

  • Sponsors
  • End users
  • Compliance and regulatory representatives
  • Legal representatives
  • Architects
  • Data Governance representatives
  • Security representatives
  • Privacy representatives

Should participate in the kick-off and the showcase sessions, where expectations are aligned on and the results of the workshop are presented respectively.

Timebox for the ideation#

The ideation happens recurringly, in parallel to the delivery. Ideation informs the delivery about what challenge needs to be addressed next. It is useful to timebox this and have the delivery team ready to start working on the implementation.

Ideally one ideation cycle takes 2-3 weeks, but the involved people should make time for it as it is an intensive process.

Additional useful techniques around ideation. What to do next?#

During the discovery and the ideation, the idea is to first explore as many options as possible. Divergent thinking is encouraged. Having many options, may also create a question of a good choice for the POs to start. When the time for decision nears, The Confidence Meter conversation can be useful to explicitly make the choice.

When the ideation is finished, and the question for "What do we think we need to build" is clarified, it is useful to align the solution team on the architecture of the solution and the technology stack. that can be employed in a feasibility assessment. This is a first detailed conversation on the solutioning and can be used for a handover after ideation.

References#