Sentiment Analysis of Oncology KOLs Has Significant Value to Pharma 

One of the most impressive aspects of the #MedTitter or "OncoTwitter" revolution has been the quality of the interaction of Key Opinion Leaders during Oncology Conferences. Through Tweets, KOLs are sharing their thoughts on clinical trials which serve as the basis for future FDA blockbuster treatments. In fact, these comments often precede the FDA approvals by 6 months to a year. Once the drug is approved, the Pharma company's entire marketing platform is based on the data from this Phase III clinical trial. The collection of these KOL sentiments will serve as a knowledge base for this FDA indication for the lifeline of the marketed drug.

One of the most interesting aspects of the discussion of Key Opinion Leaders at major oncology conferences is the sentiment analysis of their commentary. As Social Key Opinion Leaders, (or Digital Opinion Leaders), digest and share their perspectives on clinical trial data, its is important to to use the correct tools to analyze these discussions. 

KOL Sentiment Analysis - More Questions than Answers?:

Is each tweet created equal? Or are some Tweets more equal than other tweets? Also, what represents sentiment? If a clinical trial is positive(met its primary endpoint) is that always a positive sentiment? Can a negative clinical trial have a positive sentiment? Also, if a clinical trial is positive, but the tweet author shares the toxicity profile, is that negative? What about keywords and AI sentiment tools? How will they process all of these data? What about memes and colloquialisms? How can the sentiment analysis editor or AI tool process these correctly?

Sentiment Analysis is really an art form. The publisher of the tool needs to select the specific data set and convey a specific message to those consuming their data. 

What are the measurements of sentiment for Tweets?

1. You need to develop some scale.  Most AI sentiment tools will have some basic negative -neutral  -  positive framework. This may seem overly simple, but it does do a nice job of categorizing large sets of tweets. From there your editor can override the AI tools recommendation.

You can also user a STAR format made popular by YELP reviews. This is very user friendly to the readers, however, it may take some funky algorithm to translate open text tweets into a 5 star rating.

The use of a Dashboard meter may also be helpful as it is a proven user interface for comparative sentiment  analysis functions. 

2. What is positive, negative, or neutral? Even if your ratings are based solely on the  classification by a subject matter expert in Oncology, this is a tough question. This is really an art form. What are you trying to convey in your sentiment report? Is your goal to strictly rate the sentiment based on the exact words used in the tweets? Or are you taking into consideration the single tweet in the context of the entire conversation? Are you measuring the sentiment of the tweet based on the style of the tweet author? Is your goal in the sentiment analysis to highlight outliers to your readers? Measuring sentiment by Oncology KOLs is very subjective, and depends on how you want people to consume this Sentiment Analysis content. Is this tweet below from Dr. Hope Rugo at SABCS24 positive or negative?

3. What's your sentiment analysis algorithm?

This is a very important question and should accompany any Sentiment Analysis graphic. Some examples:

Basic Sentiment Meter Based on number of tweets: This is the simplest form of a KOL Sentiment Analysis tool. You treat each tweet as equal, you assign a sentiment score to each tweet, add them all up, create an average score, and make your Sentiment Meter.voila!!!! It's Clickbait Magic!!!

Advantages: It requires very minimal technology to accomplish this Sentiment Analysis tool. You just need a spreadsheet with a list of tweets, some basic sentiment scale, and maybe a template from a graphic designer, and you've got yourself a  sentiment analysis tool. You don't even need access to the X API to access any metadata about the tweets. 

Disadvantages: If you get past the reality that this is an efficient and effective clickbait tool, there are many disadvantages to this form of Sentiment Analysis:

• Each tweet has equal value: If a tweet has 100 views versus a tweet of 12,000 views, the tweets are counted equally
• Each tweet author is treated equally: If a KOL from a major Academic institution shares a criticism of the clinical trial, this tweet may be treated as equal to a tweet from some anon account trying to trade the stock of the Biotech trial sponsor.
• KOL and Physician Engagements are ignored: If the sentiment meter doesn't filter for tweet engagement(at a basic level), or more specifically engagement by other physicians or KOLs, it's really not capturing the sentiment of the crowd. It's kind of capturing gobbledygook.
• What is the reach and social graph of that tweet? This is important and the data are not available with the simple spreadsheet jockey mechanics mentioned above. When you dig into the social graphs of KOL discussions, you quickly realize that many KOLs appeal to very specific audiences. Also, the replies give you an idea of the real sentiment flowing from a given tweet and the audience of that tweet.  For example, if the Tweet author is from the US and is concerned about the cost-benefit of particular therapy, this sentiment is counterbalanced by a tweet author from Canada that just may want access to the drug (unconcerned about the US cost structure).  So what is the sentiment? Is it negative because US physicians are concerned about the cost to the system? Or is it positive because a large group of Canadian and European physicians want access to another therapy? 
• What's the Tweet style of the tweet author? Does this tweet author kind of complain all the time about this particular drug? Does this tweet author ever say anything nice? Is the tweet author a bit of a hype machine? Do they love getting clicks on their tweets by sharing positive pronouncements on clinical trials at major conferences? Is the tweet author a great networker, and loves congratulating the Lead Authors and Presenters of key clinical trial data? Are these positive sentiments of the clinical trial, or just good old fashioned flattering of people in power?
  
  

Sentiment Meter Based on Impressions: This is a decent algorithm to use which takes into account the X algo as well.  Each tweet has a basic sentiment score and there is some factor applied based on he reach of each tweet. 

Advantages: Effective algorithm that takes into consideration the actual reach of each tweet.  This requires the use of the X api to capture the impressions data on each tweet.

Disadvantages: Social Influencers via the X algorithm can really skew your results.  For example when a physician social influencer like Dr. Eric Topol tweets about an Oncology clinical trial, he'll usually gets tens of thousands of views on X. The X algo can really skew your sentiment measurement based on the causal tweets of one of two social influencers with broader reach. 

Also, this sentiment analysis is really reliant on hype at meetings. The more popular KOLS will generate a lot of Likes and Retweets, and are often not as controversial in their tweets.  As a result, the more neutral tweets shared by popular KOLs (which simply announce trial endpoints) or the popular KOLs that love to post exalting phrases can often trend the Sentiment Analysis to neutral or positive (during hype cycles).

Sentiment Meter Based on Engagements: this method uses Likes, Retweets, and Replies as the basics of measurement for the sentiment reach.  Each tweet is rated based on a sentiment scale, and then is multiplied by the engagement factor. At a higher level, confirmed engagement by other physicians, KOLs, or other only US-based physicians could be used for targeted sentiment analysis.

Advantages: Engagements will often solve the problem of the broad social influencers  that crashes the party. This tool can also provide a true measurement of the interactions of the group of physicians that are monitoring a social discussion live or remotely.

Disadvantages: The popularity of a DOL within their specific network may significantly impact their engagements.  As mentioned above, some DOLs  are excellent networkers. They take selfies at ASCO like Taylor Swift at the grammies. These folks may have a huge following of peers from their institution that Like and Retweet all of their content. The engagement measurement may really skew the importance of their sentiments. Also, physicians with a global presence may hide some important sentiments in the US. For example, if the clinical trial design used a control arm that was less than the US standard of care, but the trial presenter was from Europe, the hype of the presenter's European network may cloud over the US market concerns.

New Tools for Sentiment Analysis

Clinical Trial Timeline: This is an effective tool to measure the sentiment of tweets over time for a particular clinical trial.  Since different data sets of a clinical trial are presented at multiple meetings, and clinical trial discussions may be prompted by FDA ODAC discussions, publication in NEJM and other journals, and also on FDA approval announcements, it is helpful to see the sentiment data over time. 

These data can be shared as a collection of tweets, with overlayed impression and/or engagement data. This is a useful tool for quickly scanning the sentiment over time for a given clinical trial, and for identifying particular insights from a given discussion of a dataset. This tool may also help understand the sentiment momentum of a given drug or class of drugs as new data re presented.

Medical Conference Clinical Trial Comparison: This tool may be helpful when looking at a clinical trial sentiment analysis within the context of the other trials presented at a particular medical meeting.  When coupled with an effective "click down" UI, it's a great way for an analysts to get the vibe of a medical meetings, and to put particular sentiments in the proper perspective

Conversation Social Graph: This is a extremely effective tool for measuring the actual sentiment and reach of a particular tweet or influencer. This tool requires a command of the X API, a powerful backend, and  either python developers on call or an advanced AI tool. So basically, you're not going to fund a bunch of clickbait spreadsheet jockeys sharing these data. So why is it important. 

Targeted Sentiment: This tool tells you the particular physicians that are engaging with each influencer's tweets.This takes sentiment analysis to another level! With this tool you can understand the sentiments based on specific aspects of the digital crowd. Does a KOL's tweet impact their entire department at a major institution? Did it impact multiple institutions? Or did a negative tweet have a small reach, yet the tweet author has a huge impact at their institution

How can you apply this Sentiment Network Graph?

OMNI-Channel targeting: Run your email campaign, x ads, or even a field MSL alert based on the individual interactions of the network. 

Understand Regional Influence Trends: Based on these data you can gain a better understanding of the social influencers within institutions and also understand which physicians use social media as their education tool.

Understand Global Sentiment Dynamics: Are US physicians more positive or negative regarding a specific clinical trial design? Is the control arm of a clinical trial more relevant to European Oncologists than US Oncologists? If this is a radiation oncologist study, how does access to technology across the globe impact the trial's sentiment?

Identify Trends and Outliers: Did that grumpy physician start to grow a network? Why aren't the other KOLs interacting with this tweet author? Are community oncologists relying on specific tweet accounts? Does your MSL have any idea how critical this DOL is of your recent study?

If you are interested in applying these tools to your own sentiment analysis, please set up a demo with me!

or sign up for our SABCS24 demo to learn more