At Aurinko we get to deal a lot with the new breed of Embedded Apps and In-App Integrations, and they are very different from your pure API integrations that you could implement using iPaaS or a unified API platform. The difference is that they are not concerned with data syncing/copying but rather with UI and data presentation.
For example, you want to build an Outlook addin that could work for Salesforce CRM as well as for Sugar CRM data models, or you’re building an MS Teams bot that could show records from different CRM systems. Because such apps would be all about presenting data to an end-user they have to be able to access all objects and fields (standard, custom, related). What is the right approach here for developing more universal code? Unified API’s?
By now most developers are familiar with the Unified API concept and benefits of the platforms that provide such unification. Cloud Elements has promoted their virtualized/unified APIs since 2012 and many other companies (i.e. Kloudless, Apideck, Nylas, …) joined the camp. The pros and cons are well understood, see this article.
From all the cons listed in the article the “LCD effect” is probably quite a nuisance for more complex/custom integrations and a dealbreaker for the in-app integrations that we’re talking about as the embedded UIs usually need to show the provider data completely.
Lowest common denominator effect: If you’re choosing commonalities across many APIs, you lose intricacies of downstream payloads. A robust unified API should enable access to the full payload for power users and support custom fields.
It also seems to me that unification is based on object/field semantics, you (your code) knowing the meaning of all the unified fields. You know that ProviderA.FirstName is ProviderB.GivenName and is ProviderC.first_name, so you map those provider fields to your Virtual.firstName field and your code knows what do with that field or the mapping itself carries the sync directive (like bi-directional, read-only). If there is no analogous field for ProviderA.FieldX in ProviderB then you drop the field as it can’t be mapped/unified. Hence, the unified API’s are good for data syncing/copying integrations where all involved systems have semantically similar fields. But the in-app integrations are not about syncing and copying data between systems.
Those Office 365 addins, Zoom apps, Google Workspace addons that integrate user interface widgets may not tolerate any loss or truncation of data due to unification. These integrations don’t care about object/field semantics that much, they just have to know how to properly show all the fields and how to apply changes. Knowing just field types and relationships is good enough for them which leads us to the metadata concept.
Metadata, “data about data”
Think of data that describes your CRM object, all its fields, and relationships. Salesforce may or may not have been the pioneer of this but they have definitely done a great job on this front. Check out their sobject describe API method.
{
"name" : "Account",
"fields" :
[
{
"length" : 18,
"name" : "Id",
"type" : "id",
"defaultValue" : { "value" : null },
"updateable" : false,
"label" : "Account ID",
...
},
...
],
"updateable" : true,
"label" : "Account",
"keyPrefix" : "001",
"custom" : false,
...
"childRelationships" :
[
{
"field" : "ParentId",
"deprecatedAndHidden" : false,
...
},
....
],
"createable" : true,
"customSetting" : false,
...
}
The metadata allows you to abstract and automate the UI development. Your code does not have to know about objects and fields and their types, it simply follows the metadata descriptions and draws everything correctly according to specified field types (like textarea, currency, amount) and object attributes (like updatable, creatable). Salesforce, SugarCRM, Hubspot, Netsuite, and many other CRMs and ERPs provide metadata APIs nowadays, to full or limited extent.
Unified metadata API
Now, back to our example of building an addon that can handle multiple data providers. The metadata is what you really need to properly show and update data, and deal with all the intricacies of the provider data models. But naturally there is no standard metadata format. All those providers have different metadata formats and APIs. So we are arriving at the next level of unification - unifying the metadata APIs!
If you want to build universal UI code then you will definitely benefit from the unified metadata API just like API integrations benefit from the regular unified APIs. Data descriptions about objects and their fields have similar semantic, so this unification is quite possible. The “LCD effect” will exist here too but that is not going to affect data loss/truncation when presenting the actual data to end-users.
Imagine that Salesforce, Sugar CRM, Hubspot, Netsuite,… data models are all understood by an API platform and unified through its metadata API. Are there any platforms besides Aurinko doing that right now? Building these universal embedded UIs would become so much more automated and reliable. Agree?
There is more to the metadata topic than “data about data”. How about page layout descriptions? Anyone who has worked with Salesforce knows about those. I will write more on this topic in another blog post.