Skip to content

Data Product Registration in the Data Marketplace#

Overview#

This chapter covers the essential process of registering and publishing your Data Product in the Novo Nordisk Data Marketplace (NNDM). Registration makes your data product discoverable and accessible across the organization while establishing proper governance and ownership structures.

What you will learn#

  • How to register your Data Product in NNDM

  • How to register your Data Product in NNDM

  • Relevant roles for NNDM
  • Prerequisites and setup requirements including ServiceNow integration
  • Different registration packages and capabilities available
  • How to establish automated publishing pipelines for ongoing updates

Key Personas & Stakeholders - RACI Matrix#

Activity Product Owner Data Engineer Solution Architect Data Steward Data Owner NNDM Admin
Requirements validation A R C C C I
ServiceNow registration C R A I I C
Team setup in NNDM A I C C I A
Technical configuration C R A I I C
Metadata definition A R C C C I
Pipeline automation C R A I I I
Ongoing maintenance A R C R R C

R = Responsible, A = Accountable, C = Consulted, I = Informed

Prerequisites#

  • ✅ Completed Requirements Specification
  • ✅ Defined team structure with assigned Product Owner, Data Engineer, and Solution Architect
  • ✅ ServiceNow business application ID for hosting infrastructure
  • ✅ Git/ADO repository created with proper access controls
  • ✅ Access to NNDM and relevant organizational domains
  • ✅ Initialise the core template and the data-marketplace template following the steps mentioned in guide Set Up Repository Using Templates

Step-by-Step Process#

Key Steps for Data Product Registration#

1. Prepare Registration Information#

2. Set Up Team Structure in NNDM#

  • Contact NNDM Admin at datamarketplace@novonordisk.com to create your team with required details. Required Information:
  • Team details and descriptions
  • Business domain placement (e.g., Research, Commercial)
  • Team member role assignments (Data Owner, Data Stewards)
  • External documentation links
  • Source System/ServiceNow ID

3. Choose Registration Package#

  • Select the Base Package plus optional packages (Purview Integration or Erwin Integration).

Step-by-Step Process for Data Product Registration#

Step 1: Prepare Registration Information#

Gather required details using the Onboarding Checklist.

Step 2: Set Up Team Structure in NNDM#

Contact the NNDM Admin team at datamarketplace@novonordisk.com to create your team.

Team Configuration Requirements:

Field Description Required
Team Name Display name for your group ✅ Required
ID Unique technical identifier (UUID, URN, slug) ✅ Required
Parent Domain or Governance Group assignment ✅ Required
Type Team or Governance Group Optional
Description Detailed group description ✅ Required
Managed Tags Organizational labels for categorization Optional
Custom Fields Additional key-value metadata pairs Optional
Links References to external resources Optional

Expected Outcome: Team created in NNDM with proper domain assignment and member roles

Step 3: Choose Registration Package#

Data Product Registration Packages#

Think of our registration packages like setting up a data marketplace at Novo Nordisk. We need both the data governance system (Purview) AND the data storefront (Base Package) to successfully publish and share FAIR-compliant data products. We can then optionally add a data modeling documentation system (Erwin) for complex data architecture needs.

To publish FAIR-compliant data products in NNDM , we need:

  • Base Package (required)
  • Purview Integration (required)
  • Erwin Integration (optional)

Base Package (Required - The Data Publishing Platform)#

What it does: Enables to publish and share the data products across Novo Nordisk

Think of it as: Data marketplace(NNDM) storefront to:

  • Publish datasets as discoverable data products
  • Create data sharing agreements between departments
  • Automate secure delivery of data products to teams that request them
  • Make data discoverable and accessible to authorized users across the organization

Who needs it: Every business unit and department participating in NNDM

Purview Integration (Required - The Data Governance & Catalog System)#

What it does: Helps us to discover, catalog, understand, and govern how data is used across Novo Nordisk

Think of it as: A centralized data catalog and governance system for the entire organization to:

  • Auto-discovers and catalogs all the data sources
  • Shows where data comes from and how it flows across departments (from R&D → Clinical → Regulatory → Commercial)
  • Documents data definitions, quality metrics, and lineage
  • Tracks which teams are using which datasets and for what purpose (usage analytics)
  • Auto-classifies data by sensitivity level (confidential, regulatory, public, etc.)
  • Supports regulatory compliance reporting and audit trails
  • Maintains a single source of truth for data definitions (e.g., "what qualifies as a patient outcome?" or "how is batch quality defined?")

Why it's mandatory: We cannot publish data products without properly cataloged and governed assets. Industry best practice requires governance first, publishing second.

Who needs it: Every business unit and department participating in Novo Nordisk's data marketplace

Erwin Integration (Optional Add-on - Data Architecture & Modeling)#

What it does: Creates visual documentation of how how Novo Nordisk data is structured and organized

Think of it as: A data architecture blueprint and documentation system

  • Maps the structure and relationships between different data elements (patient IDs, batch numbers, manufacturing parameters, outcome measures, etc.)
  • Documents how data flows between legacy systems and modern cloud platforms
  • Helps IT and Data teams understand complex data models
  • Supports data standardization across Novo Nordisk
  • Useful for migration projects and system modernization initiatives

Who should consider it: Data architects and IT teams managing systems

Quick Decision Guide#

If you want to... Packages needed
Publish FAIR-compliant data products with proper governance Base Package + Purview (both mandatory)
Add advanced analytics on data usage and lineage across the organization Base + Purview + enhanced Purview analytics
Document complex data structures during system integration or acquisition Base + Purview + Erwin
Complete enterprise data governance and architecture documentation Base + Purview + Erwin

Quick Recap

Package 1 - Base Package = Mandatory (publishing platform for data products)

Package 2 - Purview Integration = Mandatory (governance and catalog system)

Package 3 - Erwin Integration = Optional (data architecture and modeling documentation)

Why 2 are mandatory:

  • We need both Base Package (to publish) and Purview (to govern and catalog assets first).
  • Industry best practice requires proper data governance before publishing.
  • Erwin is optional and recommended only for teams managing complex data architectures or handling system integrations.
  • All teams must implement Base Package + Purview together to participate in Novo Nordisk data marketplace(NNDM).

The FAIR Data Advantage#

Important

Before starting any new project at Novo Nordisk, search the Data Marketplace first.

NNDM data products follow FAIR principles—they are Findable, Accessible, Interoperable, and Reusable. Which means

  • Findable: Locate quality datasets quickly without starting from scratch
  • Accessible: Get authorized, secure access to data prepared by other teams
  • Interoperable: Use standardized data formats that work seamlessly across projects
  • Reusable: Leverage well-documented, high-quality data instead of recreating it

Business impact

Reusing FAIR-compliant data from the marketplace can save team weeks of data preparation effort while ensuring consistency and reliability across all Novo Nordisk projects. This accelerates time-to-insight and improves decision-making quality company-wide.

Expected Outcome: Package selection documented and communicated to NNDM Admin team.

4. Configure Source System Registration#

  • Ensure data sources are registered in ServiceNow. Validate against supported systems (e.g., NNEDH, Snowflake).

Step 4: Configure Source System Registration#

Ensure your data sources are registered in ServiceNow and supported by NNDM:

Supported Data Systems:

System Prerequisites
NNEDH (Enterprise Datahub) Data onboarded to NNEDH
NNEDL (Enterprise Data Lake) Data in S3 or Redshift
Snowflake dbt connectivity to ESL
AWS Glue Data Catalog Cataloged via NNEDH (recommended)
Databricks Unity Catalog Data cataloged in Unity
Microsoft Purview/Fabric No special requirements
Ab Initio No special requirements

Expected Outcome: Source systems validated and ServiceNow IDs confirmed

Step 5: Assign User Roles and Permissions#

5. Assign User Roles and Permissions#

  • Assign roles (Data Owner, Data Steward, Data Consumer) to team members via Novo Access.
Role Permissions Responsibilities
Data Owner Full data accountability Ensure data accuracy, security, policy compliance
Data Steward Edit Data Products/Contracts Manage data assets, maintain governance standards
Data Consumer Read-only access Search, request access, view models and lineage

6. Configure Automated Publishing#

  • Set up CI/CD pipeline for automatic updates.
  • For ADO: Copy pipeline code, create a .yml file, configure environment variables, and commit changes.
  • For GitHub: Create workflow file in .github/workflows/, define environment variables, and set up the workflow. | Data Consumer | Read-only access | Search, request access, view models and lineage |

Step 6: Get the Azure Container Registry (ACR) Credentials#

Contact the NNDM team at datamarketplace@novonordisk.com to request Azure Container Registry (ACR) credentials.

Required Credentials:

Credential Description Usage
ACR Username Azure Container Registry username Authentication for ADO pipeline or GitHub workflow
ACR Password Azure Container Registry password Secure access to CI tool docker images

Why These Credentials Are Needed:

The ACR credentials enable your CI/CD pipeline (Azure DevOps or GitHub) to authenticate and pull the necessary CI tool Docker images from Novo Nordisk's Azure Container Registry. These images are essential for automating the data product publishing process.

Expected Outcome: ACR credentials received and securely stored in pipeline secrets

Step 7: Configure Automated Publishing#

  • For ADO: Currently the data-marketplace template doesn't support ADO.
  • For GitHub:
    1. Initialise the templates as mentioned in pre-requisite.
    2. After initialising the project by providing the asked input parameters, push the code to your project git repository
    3. Set values of ACR_USERNAME, ACR_PASSWORD, NNDM_API_KEY in github repository secrets
    4. Make sure the repository is granted databricks access using service principal through federated access
    5. Modify the default.yaml contract generated by marketplace template under data_products/contracts/default.yaml with your data contract, or you can also upload a new data-contract file with a different name.
    6. If you replaced the default.yaml with a different filename then make sure the same detail is reflected in data_products/config.toml file in rootContract='./data_products/contracts' parameter
    7. To publish the data-contract to NNDM using CI/CD refer .github/workflows/publish_data_product.yml in dc-template-data-product template which runs the job described in .github/actions/publish_to_nndm/action.yml
    8. Make sure that the github repository have federated access granted in azure portal for the databricks environment, please refer step 7 of Set Up Azure Federated Credentials for the same

Expected Outcome: Automated pipeline configured for seamless updates

Step 8: Request Access to Data Product via Marketplace#

Once Data Products are published to the NNDM marketplace, data consumers and stakeholders can discover and request access to these products. This step facilitates the consumption phase of the Data Marketplace, allowing authorized users to:

  1. Discover published and available Data Products
  2. Review Data Product details, schemas, and contracts
  3. Request access based on business requirements
  4. Track the status of their access requests
  5. Gain access to the Data Product

1. Discover Data Products#

  • Navigate to the NNDM Data Marketplace
  • Browse the catalog of published Data Products
  • Filter by use case, business domain, or data type
  • Review Data Product descriptions, owners, and availability status

2. Review Data Product Details#

  • Examine the Data Product's data contract
  • Review schema information and data lineage
  • Check data freshness and update frequency
  • Understand SLAs and service levels
  • Identify required permissions and access levels

3. Submit Access Request#

  • Select the desired Data Product
  • Submit the access request

4. Monitor Request Status#

  • Receive notifications on request approval or rejection

5. Gain Access Upon Approval#

  • Once approved by data owners, users receive confirmation
  • Access credentials or connections are provisioned
  • Users can begin consuming the Data Product
  • Integration details are provided for downstream systems

Prerequisites

  • User account in NNDM with appropriate consumer or user role
  • Data Product must be published and available in the marketplace
  • Business justification and use case approval (if required)

Outcomes

  • Access request submitted and tracked
  • Data Product access provisioned upon approval
  • User can consume published Data Products
  • Audit trail maintained for governance and compliance

Success Metrics & Checkpoints#

  • Registration Complete: Data Product visible and searchable in NNDM
  • Team Setup: All team members have appropriate roles and access
  • Metadata Quality: All required fields populated with accurate information
  • Automation Active: CI/CD pipeline successfully publishing updates
  • Governance Compliance: ServiceNow integration and business application ID linked

Common Challenges & Solutions#

  • Challenge: ServiceNow ID not available or unclear
    Solution: Contact your IT team or ServiceNow administrators to register hosting infrastructure
    Prevention: Include ServiceNow registration in early planning phases

  • Challenge: Team members lack necessary access permissions
    Solution: Submit Novo Access requests early in the process
    Prevention: Review access requirements during team formation

  • Challenge: Source system not supported by NNDM
    Solution: Consider data migration to supported platforms or custom integration
    Prevention: Validate technical architecture against NNDM capabilities during design

Next Steps#

  1. Proceed to Data Contract Development and place it into the respective folder of the repository.
  2. Create the Data Product config.toml file and place it into the repository.

Additional Resources#