admiral

The dplyr for the pharmaceutical industry?

Stefan Thoma

Edoardo Mancini

2024-10-28

Another side to “Big Pharma”?

Section duration: 2 min

The pharmaceutical industry is comprised of hundreds of large, billion dollar companies, active in countless therapeutic areas and indications. We won’t delve into this in too much detail, but it’s safe to say that the image of “big pharma” in popular culture is one of greedy companies who put themselves and their profits ahead of the patients’ needs.

While we do believe that there is more subtlety here than meets the eye, we won’t dispute any of these claims today. Rather, in our presentation we hope to showcase a different side of big pharma - one of collaboration and cooperation towards the ultimate goal of doing more for patients around the world.

To do this, we need to establish a few key facts first:

• Historically, pharma companies have been very protective of both their clinical trial data - which makes total sense - but also their code, i.e. what they use to analyse their ongoing trials. This has resulted in lots of companies independently developing toolsets that effectively carry out very similar purposes.

• These toolsets were primarily written in SAS, a statistical software which has been industry-standard for 20+ years.

What is ADaM, and why {admiral}?

• In 2016 CDISC introduced the Analysis Data Model (ADaM). This is a therapeutic-area-agnostic set of guidelines for clinical trial dataset structure.
• ADaM lends itself to a modular tool for dataset generation.

Section duration: 4 min

In 2016, the Food and Drug Administration (FDA) mandated the use of the Analysis Data Model, also known as ADaM, in clinical trials. Without delving into too much detail, this can be thought of as a precise set of guidelines mandating the structure of the datasets that underpin clinical trial analysis. So, every company running a clinical trial after 2016 - no matter the therapeutic area - was now required to convert their collected, raw data into ADaM datasets prior to creating the tables, listings and graphs that will be used to evaluate the trial’s outcome.

Note that we are glossing over the fact that there exists an intermediate set of datasets called SDTM datasets between the raw datasets step and the ADaM step. This isn’t so relevant for our presentation, other than for the fact that it means the starting point for the ADaM datsets is also common across industry.

We also don’t need to go into too much detail about what ADaM datasets actually look like, but it’s important to understand that the steps used to construct them are often dataset-agnostic. Some examples include:

• Merging a subset of a dataset with another dataset
• Adding derived records based on existing ones
• Converting character strings to dates and imputing missing values

Consequently, it stands to reason that if ever a tool (or package) could exist to aid in the generation of these ADaMs, this tool could be modular: various derivations could be chained together and customised to generate your dataset of interest. Ideally, this package would also interact seamlessly with the {tidyverse}, so that if ever a step for adding a new variable was trivial, then simple {dplyr} code could be slotted in to your derivation sequence. This idea is exactly the thought behind the inception of the {admiral} package.

{admiral}

• Modular functions extending dplyr, and manuals and templates
  • Core functionality that can be extended to specific therapeutic areas
  • Specific to very standardized data and processes
• Created by users for users

Section duration: 2 min

In my view, admiral brings to the table three types of technical value. First and most obvious: admiral provides a modular framework for data manipulation that extends the functionalities of dplyr and is tailored to the creation of ADaM data sets. You can see on the right side an example of how this works. But the value of admiral is not just the functions. It also offers extensive manuals and vignettes that describe the programming processes and the CDISC data standard mindset behind it. In a way, it serves as an extension to the CDISC implementation guides. And to save a lot of time, it also provides templates for the different types of ADaM datasets that serve as a very practical starting point for ADaM programming.

Aside from the technical aspects, the open source nature of the project facilitated the emergence of an admiral community. This community is active both on github and on our slack channel and consists of both users and developers. The fact that these two overlap is another benefit. This allows for efficient development and very short feedback lines.

{admiral} code example

adae_dates <- ae %>%
  # join ADSL to AE
  derive_vars_merged(
    dataset_add = adsl,
    new_vars = adsl_derv_vars,
    by_vars = get_admiral_option("subject_keys")
  ) %>%
  # analysis end date/time (AENDTM), imputation flags (AENDTF/AENTMF)
  slice_derivation(
    derivation = derive_vars_dtm,
    args = params(
      dtc = AEENDTC,
      new_vars_prefix = "AEN",
      date_imputation = "last",
      time_imputation = "last",
      preserve = TRUE
    ),
    derivation_slice(
      filter = AEOUT == "FATAL",
      args = params(
        highest_imputation = "Y",
        max_dates = exprs(DTHDT)
      )
    ),
    derivation_slice(
      filter = TRUE,
      args = params(
        highest_imputation = "M",
        max_dates = NULL
      )
    )
  )

{admiral} code example

adae_dates <- ae %>%
  # join ADSL to AE
  derive_vars_merged(
    dataset_add = adsl,
    new_vars = adsl_derv_vars,
    by_vars = get_admiral_option("subject_keys")
  ) %>%
  # analysis end date/time (AENDTM), imputation flags (AENDTF/AENTMF)
  slice_derivation(
    derivation = derive_vars_dtm,
    args = params(
      dtc = AEENDTC,
      new_vars_prefix = "AEN",
      date_imputation = "last",
      time_imputation = "last",
      preserve = TRUE
    ),
    derivation_slice(
      filter = AEOUT == "FATAL",
      args = params(
        highest_imputation = "Y",
        max_dates = exprs(DTHDT)
      )
    ),
    derivation_slice(
      filter = TRUE,
      args = params(
        highest_imputation = "M",
        max_dates = NULL
      )
    )
  )

Goal of {admiral}

• Establish {admiral} workflow as the standard for ADaM creation by …

  • building trust in {admiral}
  • encouraging R adoption across the industry

• 15 pharma/biotech companies are already using {admiral}

• Part of overall effort to have an open source workflow for clinical data science, what we call the pharmaverse

Section duration: 1 min

What we would like to achieve with admiral is that it is established as the standard tool for ADaM creation in R. Not just across all therapeutic areas, but also across the pharmaceutical industry. This should help ease the transition of pharmaceutical companies towards an open source workflow for clinical data science.

For that to happen, other companies must trust the product, as switching to R is a very big challenge for companies with a history of relying on licensed analysis software. We must provide a stable solution that is accepted by regulators, and user friendly, and we must ensure that such a project will not be abandoned anytime soon.

Lets talk about how we got to that point.

Building trust

• Started as a collaboration of GSK and Roche
  • Permissive license, free to use for anyone
  • Commitment and funding from both Roche and GSK makes it hard to ignore
  • Hard for either company to monetize
  • Ensures that the package stays open source and free to use
  • Maintenance
• Open source from the start
• Involve community early on
  • Serve the needs of many, not just of one company

Section duration: 1:30 min

As trust is so important for this project, I want to share what helped us establish this in the first place. To begin with, admiral was co-developed by GSK and Roche as a collaboration. Having such large companies back the admiral project early on (i.e. assigning meaningful percentages of programmers’ time to this open sourced project) helped it to be taken seriously from the start. Very early on, we were able to ask many programmers across the industry to experiment with and give feedback on the prototype admiral package. That was very valuable, as it showed us that other companies were interested as well.

Sharing the license between two companies also makes it much harder to monetize for any company on its own. This is a good thing, as other companies probably would not like to jump on a package that might get put behind a pay wall later. These big names also help as package maintenance will come up at one point in the package life cycle, and abandoning a package is much harder if a companies reputation is somehow tied to it.

Extension packages

• Same user “feel” & language
• Same development practices
• Compatible modularity
• Stability and trust
  • Below par {admiral} extension package harms reputation of {admiral}
  • Less trust means fewer users and less adoption of open source in the industry

Section duration: 2:30 min

An extension package extends the core functionality of admiral to a specific therapeutic area. This could be for example admiralonco admiralvaccine, or admiralophtha. We want those packages to exist and be successful, but the core team does not have the capacity or the therapeutic area knowledge to take on the development of these packages. We even want them to exist for therapeutic areas where Roche & GSK are not even active yet.

Across all package extensions 9 separate companies share licensing and co-development responsibilities – and many contributors from other companies help to maintain.

We have high expectations of the extension packages: The package usage, interface and layout should look and feel coherent with the core package. Once a user is familiar with the core package, they should be able to pick up any extension package quickly. This coherence should not only be on the user level. The same should go for developers: Code should follow the same standards and practices. And it should not be redundant: If a function already exists in the core package it should not be re-implemented in the extension package. Further, all admiral packages need to be able to work together seamlessly.

Most importantly, the package needs to function reliably. Having a sub-standard extension package would reflect badly on the core package as well. This could erode trust and hinder the adoption of open source practices in the pharmaceutical industry.

High goals. But how do we get there?

Setting them up for success

• Instructions
  • {admiraldev} rules & templates
  • Git Structure & CICD
  • One core member representative
• Expectation management:
  • Small scope!
  • Effort required
  • Assigned FTE

Section duration: 2:45 min

Extension packages are developed in something like a franchise model. We took away two main points. One: We must give very clear instructions and structure to the extension developers. admiraldev is a separate package that was created specifically for admiral developers. It lists detailed instructions for code-style, naming and coding conventions, even helper functions and much more. We also provide github templates and CICD workflows that ensure certain standards like code styling, link checking, test coverage, etc. While those two things go a long way, the most resource intensive approach to this franchising is that one member of the core team acts as a representative who is either directly involved in the extension development, or is involved in a consulting function.

The second important learning is the management of expectation for the extension development teams. We have to make sure that a new team carefully considers the scope of their project. It should be well defined and achieveable. They have to think about the effort required to get there, and whether they can bring enough resources to the table to achieve that. Ideally, a certain amount of working percentages should be assigned to the task, and the workload should be shared across different companies. And most importantly, it should be clear to the team that this will require ongoing efforts: Even after most of the goals are implemented, the package needs to be maintained and periodically updated. It needs to be kept in line with the core admiral package, with the most current version of other packages it relies on, and also with changes in data standards, to remain useful to programmers.

A success story: {admiralvaccine}

• The COVID-19 pandemic was a worldwide catastrophe, but it was also an opportunity to collaborate further within pharma…
• … Enter {admiralvaccine}!

Section duration: 3 min

One of the greatest successes of our cross-company collaboration lies in the story behind the creation of an admiral extension package to deal with vaccine data.

To set the scene for you all, within the pharmaceutical industry and the wider world there was a great deal of interest in the area of vaccines during the period between 2020 and 2023 (for obvious reasons). From the outside, we all saw how quickly companies like Pfizer, Moderna and Astra Zeneca were able to bring Covid-19 vaccines to market. You will all appreciate that within the pharma industry, this required huge efforts in all areas of drug development, one of which naturally was the creation of vaccine-specific ADaM datasets.

To cut a long story short, the atmosphere of cross-company collaboration and the contribution framework that we had set up within the {admiral} family meant that we in the {admiral} family were able to connect with programmers from Pfizer, which was a company that had not meaningfully contributed to the open-source {admiral} efforts previously, and empower them take the lead on the creation of {admiralvaccine} - a vaccine-specific extension to {admiral}. The package was co-developed in partnership with two other big pharma companies GSK and Novartis, as well as members of the core {admiral} team who supported the upskilling of this new team and ensured package contents was up to par with our standards. The {admiralvaccine} package released its first version in the summer of 2023, and enabled the sponsor companies to share and open-source many of the vaccine-related learnings they had acquired during the pandemic, and we firmly believe that this will foster even more cross-industry collaborations to tackle any challenges the future might hold.

We should stress that not all aspects of this experience were strictly positive: one chief observation we made was that the team behind the vaccines package was laser-focused on releasing a first version of {admiralvaccine}, but engagement levels were lower once it came to maintaining the package. Maybe this was a by-product of the huge drive and pressure to deliver in the short term that those involved had experienced when working on the vaccines themselves. Consequently within the core {admiral} team we had to work with the {admiralvaccine} leads to convey the importance of package maintenance.

{admiral}: Looking ahead

Training models on {admiral} documentation: could genAI be the next frontier?

Section duration {1 min}

We’ve talked a lot about the past and present of {admiral}, so we’d like to conclude about what’s next for the project.

Similar to {dplyr}, the core {admiral} package has now reached a point of maturity and stability, where we do not anticipate the need for large overhauls of our material in the future. At the same time, our documentation has undergone many rounds of review and is robust and extensive. Industry uptake is constantly on the rise and we see a near future where it is the default solution for ADaM creation.

As such, one avenue for improvement in the future is with regards to the learning curve required to get used to the package. AI is all the rage right now, so why not try to apply it to {admiral} as well? At the moment, there are efforts internally at various companies to train LLMs and chatbots on {admiral} documentation, which have proven quite fruitful. There hasn’t yet been an Open Source effort in this space, though it is true that because everything is Open Source, the next re-training of something like Chat GPT may suddenly result in the model being an expert on ADaM creation!

So, we are very excited for what the future holds!

Note: could also talk about automation efforts for {admiral}, but weary of time and having to descrive things like specs first.

Q&A and resources

• {admiral}
• Cheat sheet
• {admiraldev}
• Pharmaverse blog
• Pharmaverse
• CDISC

Add: admiral website, cheat sheet, anything else? Mention the Blog as well.