It’s Your Data, Why Doesn’t it Feel That Way?
- Zohar Strinka
- Jun 2
- 7 min read
Updated: Jun 10
A guide to the challenges of working with SaaS data
When our clients buy a SaaS solution (e.g., a CRM, online advertising, et cetera), they often take for granted that they will be able to use the data from that tool when they want to. It’s something they plan to get to someday, and that day often happens to be shortly after we start working together.
Unfortunately, most of the time we learn that their data is not truly available. Or at least, it takes a lot longer and is more expensive (both to get started, and for maintenance) than we expect.
In this, the second article from our three-part series on building analytics from SaaS data, Zohar Strinka shares four surprises you might encounter as you take on the challenge. She digs into why it can be so hard to get your data and some of the issues you may run into along the way.
A note: there is a whole ecosystem of “ELT tools” that exist to make this easier. If you’re lucky, then the SaaS solution you want to pull data from has been served by an affordable ELT tool. If so, that is often your best option. Even then, it doesn’t make you immune to many of the issues listed below.
Surprise 1: They want to charge us ten thousand dollars a year to use the API connection.
When deciding if data from a SaaS vendor will be easy to get into an analytics program, the first thing we ask for at Analytics Strategies is API documentation. If it exists, we know that the SaaS company at least planned for connections from external companies in their design.
If they have an API but not documentation, expect to work closely with the vendor to figure out how to get what you need. Unfortunately, we have learned that having an API is often just the beginning of the discovery journey.
When you buy a SaaS solution, you get access to some data within the system. From a business point of view, you might assume that access to that data automatically is part of the package. But from a technical point of view, developing an API is a separate set of features and work.
Worse, APIs often provide some very useful backend data the technical folks use to make their software work. Every time the SaaS company decides to add a new feature or change how their system functions, they might tweak that backend information and break the API.
To avoid spending all their time maintaining the API, vendors often pass those changes straight through into the API as well. The changes then break downstream processes that were designed for the previous version of the API, creating problems for companies like yours consuming that data.
For the existence of an API to help you, we have to assume the way it was designed covers all the data your company wants to access. We have learned from experience that frequently only some of the data we want access to is available via the API. There are generally three reasons why.
The first is when the API was really designed to facilitate data entry into the SaaS solution, but not out of it. Think of an online advertiser that wants to make it easy for you to automate creating advertising campaigns. In these cases, the data read access (GET commands) will be a limited set that supports writing information back (PUT commands).
A second misalignment is when the API is really designed to support other partner solutions to interface with the SaaS solution provider’s data. Maybe a plugin for creating ad copy and visuals that integrate easily into the advertising platform, for instance. In those cases, the API is often set up so you have to know exactly which data you’re after.
“Pull everything” seems like a reasonable requirement, but your developers have to know how to chain together a series of requests first to learn what “everything” includes, and then request it one set at a time. It’s not insurmountable as a challenge, but behind the scenes there are also often extra permissions issues and throttling that can be difficult to recognize and resolve.
A third category of challenges is when the API is just poorly architected. It may be undocumented, incomplete, or a work in progress. With so many unique SaaS solutions, many developers have not been through the learning curve of supporting their customer’s data appetite. In some cases they are willing to learn and improve. Some companies (large and small) are surprisingly resistant to making data access a core feature of using their solution.
Navigating all these challenges presents SaaS companies with a series of headaches. We have seen every setup imaginable as a result. A fee to turn on API access. Documentation that is out of date, and surprise changes to the API. Being set up with a partner consulting org if you want to muddle your way through the issues. Incorrect or no answers to support tickets.
All the while you just want to use the data you can see in the SaaS platform to drive business value. There is no substitute for professional experience and patience when you are trying to find the best path through the technical challenges APIs can bring.
Surprise 2: We can’t just use the vendor’s snowflake connector.
What if the SaaS vendor doesn’t offer an API? Usually that means we need instead to figure out the best way to automatically generate reports and export them to online storage. Other times vendors offer an add-on solution of analytics-ready data.
The biggest gotcha when not using an API is that odds are high this is the first time anyone has chosen to grab exactly this set of data in exactly this way. That means the SaaS company may have limited suggestions about how to pull it off, and you may discover gaps in the datasets available. Depending on the SaaS company’s own analytics maturity, their suggestions often don’t follow best practices and introduce security holes or waste your team’s time trying to iterate towards something functional.
The exception to this rule is when a company has decided to productize their data access. Even then, the range in analytics expertise can be striking. SaaS companies generally won’t tell you if you are their first customer to sign up for their data share product. When you are that first customer, you can expect that not only will there be delays, but part of your role will be to serve as Quality Assurance (QA) for their new offering.
If you are fortunate enough to have a vendor that has a proven solution, you can bet that it will be expensive. At the same time, compared to all the other challenges we’ve described above, a big data product bill might be the cheapest option – as long as the data is worth it for your business.
Surprise 3: We got our data into the data warehouse, but that wasn’t enough.
Since each business is unique, most of the time you’ll run into novel problems because of how you want to use the data. The key thing that is always unique to each client is which other data they want to join to the SaaS platform data. Said another way, if all you want to do is work with the data the SaaS company already had access to, they would have had good reason to build that feature as part of the platform already.
Some SaaS vendors have figured out how important it is to be able to join their data to your other system data, and they offer mechanisms to help. For example, the ability to track a custom ID field you supply for a product instead of only using their own IDs.
Unfortunately, this solution often sends you right back to the beginning, needing to figure out how to get the right data out of the platform. When our clients have introduced customizations into the SaaS platform, we frequently find those unique fields are not available in the API or other data access options.
Another challenge we run into at the interpretation stage is needing to decode how the SaaS vendor structures their data. It is not uncommon to run into challenges even when you are analyzing only the SaaS platform data. If you think about the difficulties your team runs into analyzing just your data, you can understand why the same may be true for your vendors. If you’re lucky your vendor has documentation and built something that made a lot of sense to begin with. But the second you are trying to use the data to do something it wasn’t designed for or tested against, expect challenges.
Assuming you get to the point where the SaaS data makes sense, it’s time to figure out how to actually build the analytics that motivated the project in the first place. Since your organization is unique, you will need to figure out how to blend the SaaS platform data with your other system data to achieve the business goals that motivated the project. Frequently we discover that in order to leverage the SaaS data as well as possible, other client systems or processes need to be modified to deliver the desired ROI.
Surprise 4: We can see our whole business from here.
Once you’ve made it through the grueling process of getting access to your data and making it usable, the real fun can begin. Combining data from many sources gets you a much more complete view of your business and the opportunities to improve.
We hope this has been an informative survey of the challenges you may face when working with a SaaS vendor to gain greater access to your data. We at Analytics Strategies are happy to help you get the most out of your data. Click here to schedule an exploratory call.
Commenti