Federation matters: Introducing the NIST Cloud Federation Reference Architecture
BY DR. MARTIAL MICHEL, DR. RION DOOLEY
Data Machines Corp. (DMC) is a company that, among other things, designs cloud-based solutions for running evaluation frameworks to support research and commercial entities. On those frameworks, authorized users log onto the system using local credentials and are given access to a set of resources (compute, specialized hardware, etc) and data (including access to sequestered datasets). This is done using on-premise cloud solutions such as OpenStack or Kubernetes. Currently, Kubernetes and OpenStack are not able to communicate roles and privileges with one another; and when DMC users are in need of more resources than are locally available, bursting to public clouds needs to happen. To do so, identities and access tokens, as well as Virtual Private Connections and data copying, need to be established to enable a local user to use a remote cloud. This cloud-bursting use case happens for many companies over the world and often requires a “translation proxy” to interpret data from cloud to cloud. While technical solutions already exist to palliate this difficulty, there are no standardized means to provide such services to more users. Thus, it is the responsibility of each cloud provider to independently facilitate the sharing of identities and access to resources with the set of cloud providers it chooses to support. The ability to do this in a standardized way is the promise of Cloud Federation.
FUNDAMENTAL USE CASE
Cloud Federation can be summarized as the ability of two independent clouds to access and share resources. The typical use case follows:
Figure 1: Cloud Federation authorization flow.
A user authenticates to the Identity Provider of Organization A (OA) to establish their identity. Often, this is achieved using Identity Token exchanges; those have validity in a given cloud.
As Organization A and B have a pre-established Federation relationship, where policies and governance have been negotiated, the user is able to query the Federation Manager of Organization A (FMA) and obtain information about access to services in Organization B.
FMA provides the user’s entitlements to FMB.
FMB translates the user’s entitlements to corresponding entitlements in Organization B based on established trust agreements between the two organizations; which include an Identity Token with validity in Cloud B for the User in Cloud A (often this mapping is designed to allow the User to exist in both clouds with “equivalent” identities).
FMB returns the list of available services in Organization B to FMA.
FMA returns the list of services from FMB to the User, as well as its authorization token for Cloud B.
The User is then able to directly query the services in Organization B using its identity in Cloud B. Access to some resources is limited both by the agreements between Organizations A and B, and the access level of User A within Cloud A.
The service in Organization B validates the identity and access of User A by checking with FMB, then completes the authenticated request and uses the service requested, returning the response to the User.
ARCHITECTURE AND IMPLEMENTATION
Figure 1 describes Cloud Federation at its fundamental level. In practice, its implementations may add caching, metadata tables, pre-established trust relationships, and back-channel messaging to increase performance and resiliency within a Federation.
As such, Cloud Federation is a complex set of components and cloud computing services to match business needs: the interactions of federation models in a layered, three-plane representation of trust, security, and resource sharing and usage (see Figure 2 below). In the Trust Management Plane, Site A and Site B intend to collaborate on joint business goals and decide to establish a federation by establishing a trust relationship, defining the structure and governance of the federation they wish to create [as opposed to in the Federation Management Plane, after each Federation Operator deploys a Federation Manager (FM) that establishes secure communications between the FMs to exchange information concerning the management of federations]. In the Federation Usage Plane, Users from Site A and Users from Site B are able to access authorized services provided by Site A or Site B; this is a Virtual Administrative Domain. Maintaining the consistency and management of the users, services, policies, and authorizations of the Federated state is then an ongoing task as they could change dynamically over the course of the Federation’s lifetime.
Figure 2: Cloud Federation 3 plane architectural component view.
GOVERNANCE
Federation governance describes how the pieces of the architecture of a federation operate, work together, and interact. All federations have a set of essential characteristics that they share - from shared resources, to roles and attributes, to resource discovery, to membership, and to members’ identity credentials and terminations.
As such, as presented above, a cloud federation ecosystem is a specific configuration of semantics and governance where the formality of the ecosystem depends on the needs of its participants. Becoming a member of a federation implies providing some resources, accepting a set of rules, and controlling membership for new federation operators and members.
The benefit of cloud federation for users is important; they can get access to remote resources (data, compute, etc) that are geographically bound, and still be able to perform their analysis without having to obtain a local copy of the used dataset. This is only possible if certain rights and privileges (and in most cases, billing and accounting), are integrated within the core capabilities of the Federation model.
THE NIST CLOUD FEDERATION REFERENCE ARCHITECTURE
The NIST Cloud Federation Reference Architecture (currently an SP500 in public comment phase) defines an actor-/role-based model presenting an eleven-component model, which lend themselves to create a suite of federation options from simple to complex.
Administrative Domains (AD): essentially comprised of an Identity Provider (IdP), a Cloud Service Provider (SP), and a User, an AD is an organization wherein a uniform set of discovery, access, and usage policies are enforced across a set of users and resources based on identity and authorization credentials that are meaningful within that organization.
Regulatory Environments: legal regulations and laws that the actors in an AD must observe. A federation may need to reconcile all relevant regulatory environments.
Identity Providers: the source of the identity credentials in an AD.
Cloud Service Provider: responsible for making a cloud service available.
Cloud Service Consumer: maintains a business relationship with, and uses services from, Cloud Service Providers.
Federation Manager: provides the essential federation management functions such as Membership Management, Policy Management, Monitoring and Reporting, Accounting and Billing, and Portability and Interoperability.
Federation Operator: deploys, configures, and maintains one or more FMs.
Federation Auditor: can assess compliance for any type of policy associated with a federation.
Federation Broker: enables new members to discover existing federations.
Federation Carrier: provides connectivity and transport among federation members, or between federation consumers and federation providers.
Security: can cover the areas of identity/authentication, authorization/policy, integrity, privacy/confidentiality, and non-repudiation.
SUMMARY
The development of a standardized federation matters. The availability of standardized federation managers, interfaces, and modularized deployment are some of the tools needed to widen the availability of federation capabilities beyond the big science community, and to jump-start wider markets. Those concepts are present in the NIST reference architecture. Data Machines Corp. has been involved with this effort since 2017.
Dr. Martial Michel, DMC’s Chief Scientific Officer is one of the authors of the NIST SP500 document, the co-chair for the IEEE P2302 (Standard for Intercloud Interoperability and Federation), and a participant in conversations on this subject at international conferences (recently a panel at SuperComputing 2018).
For the last 15 years, Dr. Rion Dooley has worked towards standards-based multi-cloud and multi-institution federation. In his current role as Director of Platform Services and Solutions at Data Machines Corp., and within his affiliation with the Agave Platform project, he continues to work towards solutions that enable meaningful, unobtrusive, scientific collaborations across academic and commercial boundaries.
As private cloud providers, DMC looks forward to the availability of cloud federation for the common interfaces that will allow us to further our integration of services to support research models as well as commercial designs. As cloud consumers, we look forward to leveraging the ability to seamlessly integrate within multiple clouds. We believe this will lead to the emergence of new usage models as users are empowered with the ability to make better choices.
The NIST Cloud Federation Reference Architecture (CFRA) welcomes public comments until September 20, 2019. For more details, please see https://www.nist.gov/itl/cloud.
This work is the first step toward the development of a cloud federation standard. This effort will continue over the next few years; the development of standards happens over a period of time with the collaboration of many talented contributors. Proof of concept models might be developed by interested partners to be able to spearhead novel technical offerings in this new economy of cloud capabilities.
New collaborators are welcome: it is the work and ideas of many that help shape a better technical solution. For more information about the NIST CFRA or to join the conversation on Cloud Federation, reach out to Dr. Craig Lee (craig.a.lee@aero.org), Dr. Robert Bohn (robert.bohn@nist.gov), Dr. Martial Michel (martialmichel@datamachines.io), or Dr. Rion Dooley (riondooley@datamachines.io).
