Fogbow

Federation, opportunism and greenness in private insfrastructure-as-a-service clouds through the bartering of wares

What is Fogbow?

• Federation: aggregate resources from multiple and heterogeneous private infrastructure-as-a-service 
• Opportunism: deploy an infrastructure-as-a-service private cloud on top of shared commodity hardware in an opportunistic fashion
• Greenness: save energy by consolidating applications in active hosts and by shutting down the idle ones
• Bartering Of Wares: Trade excess resources, increasing utilization when local demand is low, and better serving your users when local demand is high

Federation requirements

• every cloud should be able to join the federation with minimal hassle
  • we don’t require an specific cloud implementation like openstack or eucalyptus
  • we need to install a federation layer in every cloud
• the cloud admin must create an user that will proxy the requests coming from the federation.
• the federation layer on every private cloud should send heartbeats to the rendezvous service
• users are registered in the identity service of the private cloud
  • as they normally are

Federation overview

Fogbow manager

• interacts with the rendezvous service and other managers
• provides an OCCI API
  • allows for requests to a large amount of instances, typically beyond what can be provided by the underlying private cloud
• requests for multiple instances are treated individually
  • each request will eventually become an instance allocation

• when do instances get allocated to pending requests?
  • while there are available resources in the underlying private cloud
  • thereafter, the remaining ones are submitted to remote members.
• open requests are rescheduled regularly until they are fulfilled or no longer valid, ie. the user has cancelled them

You can find more documentation about the Fogbow Manager at http://www.fogbowcloud.org/manager

Request: basic functioning

• type
  • one-time (default)
  • persistent
• possible states
  • open
  • failed
  • fulfilled
  • deleted
  • closed

Why Fogbow chose OCCI?

• Not specific for any private cloud technology
• Widely adopted http://occi-wg.org/community/implementations/

Federation member overview

Federation member

• the plugin layer allows for easily interaction with different underlying private clouds
  • the administrator must configure the fogbow manager accordingly
• currently, Fogbow provides Compute plugins for
  • OCCI for Openstack
  • Openstack Nova v2 
  • Open Nebula

Fogbow also provides Identity plugins for

• • Openstack’s keystone
  • Open Nebula
  • VOMS
  • x509 

Know more about Fogbow plugins at http://www.fogbowcloud.org/install-plugins

Dev challenges

• We haven’t use a reference OCCI client for starters
  • We developed the Fogbow API based on the OCCI specification and on the OCCI API for OpenStack
• We extend the private cloud’s OCCI API
  • If the underlying cloud already provides an OCCI API, Fogbow will extend it with its own features
• Bypass operations not supported by Fogbow to the private cloud OCCI API
  • OCCI requests that shouldn’t be treated by Fogbow must be bypassed to the underlying private cloud OCCI interface (if it exists)
    • E.g. the creation of an instance - Fogbow allows for the creation of requests but won’t allow for instance creation directly. If an user tries to create instances on the Fogbow endpoint, it should be bypassed to private cloud.

What happened when we tried to run rOCCI client on top of Fogbow?

We found out that Fogbow Manager wasn’t fully OCCI compliant
We also find a bug at rOCCI https://github.com/EGI-FCTF/rOCCI-cli/issues/7

Bugs fixed in Fogbow

authentication required, when it wasn’t
authentication not required, when it was
header required, when it wasn’t
wrong location format of created requests
support of multiple values in the same header
“Accept” header with “text/uri-list” wasn’t supported
“Accept” header being ignored when bypassed
“occi.core“ attributes weren’t supported
get request output format
query interface output format
some categories weren’t added on this return
www-authenticate header wasn’t added in the unauthorized responses

Learning

• Initially we thought it was possible to develop only one OCCI compute plugin that cloud be used on top of any private cloud technology, but it isn’t the case different OCCI implementations can interact differently with their users (for instance, the way how request content must be passed – header or body of message)
• It was a great experience to have an OCCI client to run on top of the Fogbow Manager
• It helped us to find some simple bugs
• We improved our understanding about OCCI through the use of this client
• A create request message with “org.fogbowcloud.request.instance-count=x” attribute, where x > 1, return “x” request locations
• We aren’t sure if that is the better way to implement it

Fogbow data for testing

Fogbow endpoint http://150.165.15.107:8182
cloud credentials:
user: cloudplugfest
password: cloudplugfest

Thank you!

You can find more information about Fogbow at http://www.fogbowcloud.org

Feel free to contact us

• fogbow-dev-l@lsd.ufcg.edu.br
• giovanni@lsd.ufcg.edu.br
• abmargb@gmail.com