Build vs Buy: Authentication and Identity Systems
Monday, October 28, 2024
6 min read
There's a prevailing wisdom in software development that teams should never build their own authentication and identity systems, instead opting for Commercial Off-the-Shelf (COTS) solutions. This is quite a shift in thinking within the industry, where historically companies have built these systems in-house, supported by frameworks/platforms which typically shipped with libraries that simplify the task. For example .NET Core ships Identity as a solution to the task of maintaining user identities.
Challenging the Status Quo
It seems that the motivater in most of the recent messaging is that development teams and organizations are not capable of implementing a proper authentication solution, nor maintaining it properly going forward. This perspective is not only flawed but potentially harmful to our industry. It perpetuates a misconception that authentication is too complex a problem to understand or implement, leading to:
- A sense of technical inadequacy among development teams
- Concentration of risk in a few dominant identity management platforms
- Oversimplification of what should be a nuanced business decision
The reality is that the likelihood of security breaches or regressions isn't inherently higher with in-house solutions compared to third-party platforms. The primary difference lies in responsibility and liability allocation. This should frame it as a business decision rather than a purely technical one.
Cases for Building In-House
So in what situations does it make sense for a business or team to roll their own auth and build in-house? A few stand out:
1. Technical and Feature Requirements
Many enterprises face specific technical constraints that make custom solutions necessary:
- Legacy system integration requiring specific protocols
- Custom session management requirements
- Network constraints affecting authentication flows
- Fine-grained control needs for distributed state management
- Complex business processes requiring specific technical implementations
Plenty of complex and interesting use cases are out there, and we should not be limited by the tools that we use. Unfortunately with such core activities like authentication, discovering short comings of any COTS solution can come late in the game and be very costly. It's crucial to thoroughly evaluate your domain-specific requirements before committing to any solution, and considering in-house an option will broaden your perspective.
2. Data Ownership and Modeling
Another likely situation you may find yourself in is that you (or your customers) have specific requirements about data governance and third party services may not offer the necessary features to enable you to satisfy these requirements:
- Regulatory compliance needs
- Data sovereignty requirements
- Custom data modeling requirements
- Complex data relationship mapping
This can be a complete blocker to adopting a third party system, but also highlights the trade-off being made: You are trusting the third-party entity with core data which powers your platform. I want to highlight this as it can be taken for granted easily, and understanding the risk is crucial to making informed business decisions.
In addition to added risk, there is the added cost of checking in with your vendor on a regular basis to ensure that they are holding up to their end of the bargain. This can be a hidden cost because it becomes part of "maintenance" work that your engineering or trust teams take on as part of compliance measures. There is also the case of the customer success and sales teams that will need to stitch together the picture for any of your customers that inquire about your data governance and authentication practices.
Looking past the logistics around data storage and handling, we also make a tradeoff in the flexibility of our data modeling when signing up with a third-party vendor. Simple limitations like custom fields of specific types may be quick to discover, but more complex model relationships and transformations required to integrate with your business successfully might be harder to discover without running a proof of concept or trial.
3. Long-Term Cost Considerations
Upfront cost of building, testing, and integrating the system will be high -- quite high depending on the size and complexity of your system. But while that is true, in the long run you are taking control of the cost and pricing of running this core part of your system. Down the line a vendor might increase prices or move features that you rely on into "add-ons", worse yet features may have limitations placed on them that could make life a lot harder. We simply cannot predict these decisions, nor predict motivators that might come along. This predictability in cost and capability has tangible value for teams and businesses, and should not be discounted.
There is also something to be said about reducing your ongoing licensing fees. In the end how much will be spent on licensing fees, support hours, implementation and maintenance? Over time will this cost surpass that of building in-house? I suspect this is true, just the time frame of the intersection of costs might be pushed out due to competitive pricing of market dominators.
Cases for Off-the-Shelf Solutions
Now that we have looked at a few key reasons why you might make the decision to build in-house, let's flip the coin and take a look at why you might make the decision to go with a COTS solution:
1. Implementation Complexity and Commitment
The protocols that power most authentication on the internet are canonized standards that are well documented, and well tested in the real world. Although this is true, implementation of the standards requires meticulous review and understanding to ensure that you are deploying a secure and accurate experience. For example the OAuth 2.0 spec (RFC 6749) documents multiple different flows for implementation along with their appropriate use. It takes careful consideration and more reading to discover what is needed for your particular business use cases, for example PKCE (Proof Key for Code Exchange) has it's own lengthy spec (RFC 7636) which one needs to review in order to understand and implement properly.
This is only the tip of the iceberg, so teams need to be sure they are willing (and able) to put in the effort to properly consider, document, test, and implement the necessary standards and flows correctly. It will be hard to pivot later and costly if errors are found out too late in production, potentially causing your users and business a great deal of damage. Even beyond just the necessary care to implement the specs properly, you must also ensure that you are building services that meet your business needs. Achieving the necessary scale and efficiency that is expected of all of your services is crucial to ensuring success. This is a responsibility that a third-party vendor takes off of your hands, allowing your business to focus on the core use cases of your product(s).
2. Ongoing Maintenance and Evolution
Another hidden cost to a mission critical system like authentication is that the landscape is constantly evolving. As new exploits are discovered, hardware gets more powerful, and new authentication features are developed, you will want to provide the best experience for your users and so you will need to stay up to date with these changes:
- Implementation of new authentication methods (e.g., passkeys)
- Regular security updates
- Encryption best practices
- Protocol updates and deprecations
- Performance optimizations
Picking an off-the-shelf solution allows you to not only gain these new features with little to no effort, but also keep up with best practices behind the scenes so that you are never sacrificing on your end users experience or security. If your organization does not have great process for platform maintenance, or does not treat tech debt as a first class citizen you may be better off opting for one of the off-the-shelf solutions to power your identity and authentication.
Making the Decision
The choice between building in-house and purchasing a COTS solution should be guided by what best serves your customers, teams, and business objectives. Avoid oversimplified decision-making driven by marketing narratives. Instead:
- Evaluate your specific requirements thoroughly
- Consider both immediate and long-term implications
- Assess your team's capabilities and resources
- Factor in maintenance and evolution costs
- Consider your organization's risk tolerance and compliance needs
Remember that either choice can be valid -- the key is making an informed decision based on your specific context rather than following industry trends blindly.
Was this helpful?
