From Zero to Hero with the K5 OpenStack IaaS API (Python 2.7X)

Today I’ll go through all the steps necessary to deploy a server on Fujitsu K5’s IaaS platform using only OpenStack based API calls. I’ve chosen Python 2.7X for demonstration purposes as this is what Openstack itself is mainly built on. However, it should be easy enough to port the API calls to another scripting language as I’m not using any intermediary libraries – all my http requests talk directly to K5 endpoints.

Prerequisites for this tutorial:

  • A valid K5 account with access to a project or a valid OpenStack account that utilises Keystone V3 with Domains and access to a project.
  • The account must be a project administrator – i.e. you need to have the roles to build and delete resources within the project
  • A windows or linux system with python 2.7X installed along with the Requests module.

Steps required to build a server using the OpenStack API

Once you’ve satisfied the above prerequisites let’s quickly review what’s needed to build a server in a project within OpenStack.

k5-iaas-server-build-via-api-1

For the remainder of this tutorial I’ll step through each API call detailing what is submitted and what gets returned.

sbstep1

Step1 – Get a valid K5 token scoped to the project where you wish to deploy the server. This is where you’ll need your K5/OpenStack login credentials. These will be supplied with the initial request for a project scoped token as follows:

def get_scoped_token(adminUser, adminPassword, contract, projectid, region):

    identityURL = 'https://identity.' + region + \
        '.cloud.global.fujitsu.com/v3/auth/tokens'

    try:
        response = requests.post(identityURL,
                                 headers={'Content-Type': 'application/json',
                                          'Accept': 'application/json'},
                                 json={"auth":
                                         {"identity":
                                          {"methods": ["password"], "password":
                                           {"user":
                                           {"domain":
                                               {"name": contract},
                                            "name": adminUser,
                                            "password": adminPassword
                                            }}},
                                          "scope":
                                          {"project":
                                           {"id": projectid
                                            }}}})

        return response
    except:
        return 'Regional Project Token Scoping Failure'



def main():

  k5token = get_scoped_token(adminUser, adminPassword, contract, demoProjectAid, region).json()
  # print the output for demonstartion purposes
  pprint.pprint(k5token)

 

If valid credentials have been supplied then you’ll get a JSON response object that contains the K5 endpoints registered in its OpenStack keystone catalog as follows:

{u'token': {u'catalog': [{u'endpoints': [{u'id': u'b506af0e016a4b5fb592f196da569a41',
                                          u'interface': u'public',
                                          u'name': u'objectstorage',
                                          u'region': u'uk-1',
                                          u'url': u'https://objectstorage.uk-1.cloud.global.fujitsu.com/v1/AUTH_7015d1478a4c4bd7b970215d7b0260dd'}],
                          u'id': u'024329e4fde148e58ecd90b7d9872438',
                          u'type': u'object-store'},
                         {u'endpoints': [{u'id': u'0419c448001845af8f6828cf49745e72',
                                          u'interface': u'public',
                                          u'name': u'keymanagement',
                                          u'region': u'uk-1',
                                          u'url': u'https://keymanagement.uk-1.cloud.global.fujitsu.com/v1'}],
                          u'id': u'07f309b0ef9d42758ea4de47bdca9c32',
                          u'type': u'keystore'},
                         {u'endpoints': [{u'id': u'e1cc93936fb94cdbadc20f17c4ad3140',
                                          u'interface': u'public',
                                          u'name': u'certificate',
                                          u'region': u'uk-1',
                                          u'url': u'https://certificate.uk-1.cloud.global.fujitsu.com/v1'}],
                          u'id': u'0bd9a971e97d4c15af6b94311e4e9c15',
                          u'type': u'certificate'},
                         {u'endpoints': [{u'id': u'c3ee708a6ae24ddf92c078526c36a446',
                                          u'interface': u'public',
                                          u'name': u'orchestration',
                                          u'region': u'uk-1',
                                          u'url': u'https://orchestration.uk-1.cloud.global.fujitsu.com/v1/7015d1478a4c4bd7b970215d7b0260dd'}],
                          u'id': u'0dafa2a9240c41268c37818979769c88',
                          u'type': u'orchestration'},
                         {u'endpoints': [{u'id': u'ba9061d324954b1a9f6b1c1f5d4a5c5e',
                                          u'interface': u'public',
                                          u'name': u'blockstorage',
                                          u'region': u'uk-1',
                                          u'url': u'https://blockstorage.uk-1.cloud.global.fujitsu.com/v1/7015d1478a4c4bd7b970215d7b0260dd'}],
                          u'id': u'0ec640e57e4d4374841d2ab7b292f2c2',
                          u'type': u'volume'},
                         {u'endpoints': [{u'id': u'169bcb3f92384f709232864f4d5304eb',
                                          u'interface': u'public',
                                          u'name': u'loadbalancing',
                                          u'region': u'uk-1',
                                          u'url': u'https://loadbalancing.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'15ef761fa00a438985213aec7b6fb18a',
                          u'type': u'loadbalancing'},
                         {u'endpoints': [],
                          u'id': u'1a53a2e82370422d90174c4acf585ea9',
                          u'type': u'dwh'},
                         {u'endpoints': [],
                          u'id': u'1b35fa74ae0e46ebbc2014f976fec0e1',
                          u'type': u'contract'},
                         {u'endpoints': [],
                          u'id': u'253096ed430c49c48c9f3549e867b538',
                          u'type': u'appplatform'},
                         {u'endpoints': [{u'id': u'e26ad89a682d4c8f8a71c9e76cd785b4',
                                          u'interface': u'public',
                                          u'name': u'compute-w',
                                          u'region': u'uk-1',
                                          u'url': u'https://compute-w.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'327c11710f184555b43bc6fdfc47626b',
                          u'type': u'compute-w'},
                         {u'endpoints': [],
                          u'id': u'3435ad7a1da04b1cb2e249665529eb51',
                          u'type': u'notification'},
                         {u'endpoints': [{u'id': u'f70e51b9a6a74a87a4c7055b8df8bedf',
                                          u'interface': u'public',
                                          u'name': u'compute',
                                          u'region': u'uk-1',
                                          u'url': u'https://compute.uk-1.cloud.global.fujitsu.com/v2/7015d1478a4c4bd7b970215d7b0260dd'}],
                          u'id': u'3e19093f50cc4590973c8953e2c327f9',
                          u'type': u'compute'},
                         {u'endpoints': [{u'id': u'c46b5ec51ab04402a9ffd8177743ba6f',
                                          u'interface': u'public',
                                          u'name': u'queue',
                                          u'region': u'uk-1',
                                          u'url': u'https://queue.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'4a9ab69b74ec4948a0a6958177d30387',
                          u'type': u'queue'},
                         {u'endpoints': [{u'id': u'071ee50be4b341558e37f84abac47d02',
                                          u'interface': u'public',
                                          u'name': u'autoscale',
                                          u'region': u'uk-1',
                                          u'url': u'https://autoscale.uk-1.cloud.global.fujitsu.com/autoscale_schedulers'}],
                          u'id': u'4f158b2836434c7ca6ee47e40a3ee56c',
                          u'type': u'autoscale'},
                         {u'endpoints': [{u'id': u'6ac15657fbc548d7ab2ed986b8d94192',
                                          u'interface': u'public',
                                          u'name': u'telemetry',
                                          u'region': u'uk-1',
                                          u'url': u'https://telemetry.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'54ee7596f0e1433cb8db6201be2cf772',
                          u'type': u'metering'},
                         {u'endpoints': [],
                          u'id': u'5954129d136e4a76b7cd6f3d1f3808b5',
                          u'type': u'cdn'},
                         {u'endpoints': [],
                          u'id': u'5a252878ffd7402498b6b74b575a899a',
                          u'type': u'oss'},
                         {u'endpoints': [{u'id': u'2d600a5de6ec42c6a5eab7b519832d87',
                                          u'interface': u'public',
                                          u'name': u'rolemanagement',
                                          u'region': u'uk-1',
                                          u'url': u'https://rolemanagement.uk-1.cloud.global.fujitsu.com/v1'}],
                          u'id': u'5bc90099f0ad490c811239a6a3a8f853',
                          u'type': u'rolemanagement'},
                         {u'endpoints': [{u'id': u'768450282d5546409395d2cb35502dad',
                                          u'interface': u'public',
                                          u'name': u'compute-b',
                                          u'region': u'uk-1',
                                          u'url': u'https://compute-b.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'65f754202b6049578e9a292a801e28f7',
                          u'type': u'compute-b'},
                         {u'endpoints': [{u'id': u'675910513a7e466c84c39c5ca9445b81',
                                          u'interface': u'public',
                                          u'name': u'networking',
                                          u'region': u'uk-1',
                                          u'url': u'https://networking.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'6cf37e33dc7140b9b032faabfa53a53e',
                          u'type': u'network'},
                         {u'endpoints': [],
                          u'id': u'7426c97af5174c1abdb8b84ba02227ce',
                          u'type': u'dns'},
                         {u'endpoints': [{u'id': u'6976cbeed14c464da15c13e949b41607',
                                          u'interface': u'public',
                                          u'name': u'database',
                                          u'region': u'uk-1',
                                          u'url': u'https://database.uk-1.cloud.global.fujitsu.com/v1.0/7015d1478a4c4bd7b970215d7b0260dd'}],
                          u'id': u'75c0280f15844d7a9eb22b1ad5a978e8',
                          u'type': u'database'},
                         {u'endpoints': [{u'id': u'db15a7126d4a4f1f81903bb4d56be32b',
                                          u'interface': u'admin',
                                          u'name': u'identity',
                                          u'region': u'uk-1',
                                          u'url': u'https://identity.uk-1.cloud.global.fujitsu.com/v2.0'}],
                          u'id': u'79066fe949064550930a61b33a49eb53',
                          u'type': u'identity'},
                         {u'endpoints': [],
                          u'id': u'804190a423804019aab8fd934db0086e',
                          u'type': u'intdns'},
                         {u'endpoints': [],
                          u'id': u'8d364df08cbc4d7188f0cf83744b0336',
                          u'type': u'invoicing'},
                         {u'endpoints': [],
                          u'id': u'9674ae5917af401397071c028ff33b68',
                          u'type': u'catalog'},
                         {u'endpoints': [{u'id': u'fd6g35fgknrzumut7bx6p6bwzehkfhrz',
                                          u'interface': u'public',
                                          u'name': u'software',
                                          u'region': u'uk-1',
                                          u'url': u'https://software.uk-1.cloud.global.fujitsu.com/v1.0'}],
                          u'id': u'9760973e49b04b00b41e60c180a80562',
                          u'type': u'software'},
                         {u'endpoints': [{u'id': u'8c4ace3bbbd44883b5762a9ad8432a46',
                                          u'interface': u'public',
                                          u'name': u'blockstoragev2',
                                          u'region': u'uk-1',
                                          u'url': u'https://blockstorage.uk-1.cloud.global.fujitsu.com/v2/7015d1478a4c4bd7b970215d7b0260dd'}],
                          u'id': u'9bf82d0466714eee86e304ab3301aec9',
                          u'type': u'volumev2'},
                         {u'endpoints': [],
                          u'id': u'a2b90094346c48deabb9ca645bacf588',
                          u'type': u'dnsadmin'},
                         {u'endpoints': [{u'id': u'225257ec50ef47668185587cc6edb95d',
                                          u'interface': u'public',
                                          u'name': u'vmimport',
                                          u'region': u'uk-1',
                                          u'url': u'https://vmimport.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'c73fa07c3b604b4f89d5a2b9c4621cf2',
                          u'type': u'vmimport'},
                         {u'endpoints': [{u'id': u'27dc2bba1c5d4a14b68657fc8fdd4e3e',
                                          u'interface': u'public',
                                          u'name': u'identityv3',
                                          u'region': u'uk-1',
                                          u'url': u'https://identity.uk-1.cloud.global.fujitsu.com/v3'}],
                          u'id': u'cc6f50d496884ef0a751acb2e1eceedd',
                          u'type': u'identityv3'},
                         {u'endpoints': [{u'id': u'06fbcf8235434f23a08faed03b4af9ac',
                                          u'interface': u'public',
                                          u'name': u'appmanagement',
                                          u'region': u'uk-1',
                                          u'url': u'https://applicationmanagement.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'ce2f6bbe7a9f446b82d63725bd7bb484',
                          u'type': u'appmanagement'},
                         {u'endpoints': [],
                          u'id': u'd07aa581c0734b6bb85cd496115b5110',
                          u'type': u'nosql'},
                         {u'endpoints': [],
                          u'id': u'd2f57dcae4c84466ac357220ff3d8900',
                          u'type': u'baremetal'},
                         {u'endpoints': [],
                          u'id': u'd4601f51af8f43f3b0e3409e4cbac690',
                          u'type': u'mail'},
                         {u'endpoints': [{u'id': u'bf356065e45349a5bde5e043c95a1923',
                                          u'interface': u'public',
                                          u'name': u'networking-ex',
                                          u'region': u'uk-1',
                                          u'url': u'https://networking-ex.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'd4937aa1275c41378bf2ae1dbe829c68',
                          u'type': u'networking-ex'},
                         {u'endpoints': [{u'id': u'dc2ebf9d6fc04f1facc263a733cf754d',
                                          u'interface': u'public',
                                          u'name': u'global-identity',
                                          u'region': u'jp-east-1',
                                          u'url': u'https://identity.gls.cloud.global.fujitsu.com/v3'}],
                          u'id': u'e74b9c29d6504f10a8fb8e1495f3f5c6',
                          u'type': u'global-identity'},
                         {u'endpoints': [{u'id': u'a832d640886b4ef89027339a2edf8fcd',
                                          u'interface': u'public',
                                          u'name': u'image',
                                          u'region': u'uk-1',
                                          u'url': u'https://image.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'ea850dab0e964e12a9f66787ee8623ae',
                          u'type': u'image'},
                         {u'endpoints': [],
                          u'id': u'f1302a2477bf492bb3e867952175974a',
                          u'type': u'billing'},
                         {u'endpoints': [{u'id': u'e8bde8fe61a14a88b414e5568fc17201',
                                          u'interface': u'public',
                                          u'name': u'import-export',
                                          u'region': u'uk-1',
                                          u'url': u'https://import-export.uk-1.cloud.global.fujitsu.com'}],
                          u'id': u'fc302725919645199373077fa299fc6e',
                          u'type': u'import-export'}],
            u'expires_at': u'2017-01-27T13:53:55.982381Z',
            u'extras': {},
            u'issued_at': u'2017-01-27T10:53:55.982425Z',
            u'methods': [u'password'],
            u'project': {u'domain': {u'id': u'3256d41b17014d5c99727993d6fca821',
                                     u'name': u'YssmW1yI'},
                         u'id': u'7015d1478a4c4bd7b970215d7b0260dd',
                         u'name': u'Project_A'},
            u'roles': [{u'id': u'0739580a550d4a0f9c78f45a9f038c05',
                        u'name': u'cpf_systemowner'},
                       {u'id': u'3af119c426a742999e7890f6d1f70b36',
                        u'name': u'cpf_admin'}],
            u'user': {u'domain': {u'id': u'3256d41b17014d5c99727993d6fca821',
                                  u'name': u'YssmW1yI'},
                      u'id': u'03758af9df694db784c3547ab8217abb',
                      u'name': u'landg'}}}

 

This information is very important as when you wish to fully automate K5 this is how you detect your various endpoints for the services that you need to consume.

Note: Endpoints that have a ‘-ex’ suffix in their names are the OpenStack enhanced endpoints. These endpoints include the additional features that Fujitsu have added to K5 above and beyond the standard OpenStack features.

So what’s missing from this picture….where’s the token??? Fear not it’s actually stored in the header of the response in a variable called ‘X-Subject-Token’. For example – 

  # print the K5 poject scoped token
  print "\n\n K5 Project Scoped Token :", k5token.headers['X-Subject-Token'], "\n"

 

…reveals a token, which by the way is valid for 2 hours, like so..

K5 Project Scoped Token : f5b72cd06517447fae59360f6deba7d8 

 

This token must be passed with all the subsequent K5 API calls in the header to provide your authentication and authorisation credentials to each service request.

The standard OpenStack neutron api endpoint can be extracted from the above response by looking for the endpoint called ‘networking’. I’ve used the following function to return the URL of the endpoint that I’m looking for –

def get_endpoint(k5token, endpoint_type):
    # list the endpoints
    for ep in k5token.json()['token']['catalog']:
        if len(ep['endpoints'])>0:
            # if this is the endpoint that  I'm looking for return the url
            if endpoint_type == ep['endpoints'][0].get('name'):
                #pprint.pprint(ep)
                return ep['endpoints'][0].get('url')

 

For example –

  # list the networking endpoints
    print get_endpoint(k5token,"networking")

 

Will return the neutron API endpoint that I need –

 K5 Networking EndPoints :
https://networking.uk-1.cloud.global.fujitsu.com

 

Now that we have the necessary token and endpoint details let’s move to the next stage.

sbstep2

Step 2 – Create the layer 2 network. The following neutron call creates a K5 layer 2 subnet with the properties that you pass via the API as follows:

def create_network(k5token, name, availability_zone):


    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/networks')
    print networkURL
    token = k5token.headers['X-Subject-Token']
    try:
        response = requests.post(networkURL,
                                 headers={'X-Auth-Token': token,
                                         'Content-Type': 'application/json'},
                                 json={
                                            "network":
                                            {
                                              "name": name,
                                              "admin_state_up": True,
                                              "availability_zone": availability_zone
                                             }
                                        })
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

For example –

    # Create a layer 2 virtual network
    network = create_network(k5token, "demonet", "uk-1b")
    print network
    print network.json()

 

produces –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/networks

{u'network': {u'status': u'ACTIVE', u'subnets': [], u'name': u'demonet', u'admin_state_up': True, u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'availability_zone': u'uk-1b', u'shared': False, u'id': u'02fd7be4-b5f1-455c-8aea-37b299e85ddd'}}

 

We need to capture the id of the newly created network as we’ll use it in the following API call –

    network_id = network.json()['network'].get('id')

 

Now let’s create our subnet.

sbstep3

Step 3 – Create the layer 3 subnet. Again we’re using the standard neutron API call here and passing the newly created network id in to the call. The function used looks like this –

def create_subnet(k5token, name, netid, cidr, az):


    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/subnets')
    print networkURL
    token = k5token.headers['X-Subject-Token']
    try:

        response = requests.post(networkURL,
                                headers={'X-Auth-Token': token,
                                         'Content-Type': 'application/json'},
                                json={
                                             "subnet": {
                                                 "name": name,
                                                 "network_id": netid,
                                                 "ip_version": 4,
                                                 "cidr": cidr,
                                                 "availability_zone": az
                                             }
                                            })
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

And when called as follows –

    subnet = create_subnet(k5token, "demosubnet", network_id, "192.168.10.0/24", "uk-1b")

    print subnet

    print subnet.json()

    subnet_id = subnet.json()['subnet'].get('id')

    print subnet_id

 

produces the following response –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/subnets

{u'subnet': {u'name': u'demosubnet', u'enable_dhcp': True, u'availability_zone': u'uk-1b', u'network_id': u'1af9e3fe-4546-4b59-a5a5-10e48af5302a', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'dns_nameservers': [], u'allocation_pools': [{u'start': u'192.168.10.2', u'end': u'192.168.10.254'}], u'host_routes': [], u'ip_version': 4, u'gateway_ip': u'192.168.10.1', u'cidr': u'192.168.10.0/24', u'id': u'2322908f-cf58-4124-a273-86a33cc429b5'}}
2322908f-cf58-4124-a273-86a33cc429b5

 

Note that we’ve now also captured the new subnet id. Hopefully the pattern is becoming obvious now as we move on to the next step.

sbstep4

Step 4 – Create a virtual router.Yet again we’re working with OpenStack neutrons endpoint this time building a virtual router. The following function has been used –

def create_router(k5token, name, availability_zone):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/routers')
    print networkURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.post(networkURL,
                                headers={'X-Auth-Token': token,
                                         'Content-Type': 'application/json'},
                                json={
                                          "router": {
                                               "name": name,
                                               "admin_state_up": True,
                                               "availability_zone": availability_zone
                                          }})
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

which, when invoked like this –

    # Create a layer 3 virtual router
    router = create_router(k5token, "demorouter", "uk-1b")

    print router

    print router.json()

    router_id = router.json()['router'].get('id')

    print router_id

 

returns –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/routers

{u'router': {u'status': u'ACTIVE', u'external_gateway_info': None, u'name': u'demorouter', u'admin_state_up': True, u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'availability_zone': u'uk-1b', u'id': u'0a0661a9-7e30-4f8e-904a-c4a9469d3fca'}}
0a0661a9-7e30-4f8e-904a-c4a9469d3fca

 

Once again we capture the newly created resource identifier – router_id.

sbstep5

Step 5 – Now let’s configure the router. First we set the external gateway using the following function call –

def update_router_gateway(k5token, router_id, network_id):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/routers/') + router_id
    print networkURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.put(networkURL,
                                headers={'X-Auth-Token': token,
                                         'Content-Type': 'application/json'},
                                json={
                                         "router": {
                                                     "external_gateway_info": {
                                                                                    "network_id": network_id
                                                     }
                                         }})
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

called from the script body like so –

    # Add external gateway to router (extaz2 is the external network id for the external network in availability zone b)
    router_gateway = update_router_gateway(k5token, router_id, extaz2)

    print router_gateway

    print router_gateway.json()

 

producing the following results –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/routers/245a02e7-8b0e-4d1f-af6a-dfc07ed916e0

{u'router': {u'status': u'ACTIVE', u'external_gateway_info': {u'network_id': u'd730db50-0e0c-4790-9972-1f6e2b8c4915', u'enable_snat': True}, u'name': u'demorouter', u'admin_state_up': True, u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'availability_zone': u'uk-1b', u'routes': [], u'id': u'245a02e7-8b0e-4d1f-af6a-dfc07ed916e0'}}

 

sbstep6

Step 6 – We will now ‘plug’ our virtual network into our virtual router with this function call –

def add_interface_to_router(k5token, router_id, subnet_id):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/routers/') + router_id + '/add_router_interface'
    print networkURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.put(networkURL,
                                headers={'X-Auth-Token': token,
                                         'Content-Type': 'application/json'},
                                json={
                                    "subnet_id": subnet_id})
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

which is invoked like this –

    # Plug new network subnet into the router
    router_interface = add_interface_to_router(k5token, router_id, subnet_id)

    print router_interface

    print router_interface.json()

 

with the following output –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/routers/43e95138-9124-4e04-a5d4-ae5b90f4fded/add_router_interface

{u'subnet_id': u'16b59543-c303-4e33-9b96-b538625f40b2', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'port_id': u'1ba71079-288e-49b7-925d-fb31658bc9c6', u'id': u'43e95138-9124-4e04-a5d4-ae5b90f4fded', u'availability_zone': u'uk-1b'}

 

This completes the networking infrastructure basics.

sbstep7

Step 7 – The server will need a security group to be assigned to it network interfaces so we’ll create this now along with the rules that will allow ssh, tcp and rdp protocols inbound only.

This API call was used to create the security group –

def create_security_group(k5token, name, description):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/security-groups')
    print networkURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.post(networkURL,
                                headers={'X-Auth-Token': token, 'Content-Type': 'application/json', 'Accept': 'application/json'},
                                json={
                                        "security_group": {
                                            "name": name,
                                            "description": description
                                            }
                                        })
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

which is invoked as follows –

    # Create a new security group
    security_group = create_security_group(k5token, "demosecuritygroup", "Demo Security Group Allows RDP, SSH and ICMP")

    print security_group

    print security_group.json()

    security_group_id = security_group.json()['security_group'].get('id')

    print security_group_id

 

resulting in this output –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/security-groups

{u'security_group': {u'id': u'ae44389e-f884-4726-8bc3-ec3cbc7b876e', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'description': u'Demo Security Group Allows RDP, SSH and ICMP', u'security_group_rules': [{u'remote_group_id': None, u'direction': u'egress', u'protocol': None, u'ethertype': u'IPv6', u'port_range_max': None, u'security_group_id': u'ae44389e-f884-4726-8bc3-ec3cbc7b876e', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'port_range_min': None, u'remote_ip_prefix': None, u'id': u'55d5c3f1-3cf0-46c0-b876-6e89a1a1fc55'}, {u'remote_group_id': None, u'direction': u'egress', u'protocol': None, u'ethertype': u'IPv4', u'port_range_max': None, u'security_group_id': u'ae44389e-f884-4726-8bc3-ec3cbc7b876e', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'port_range_min': None, u'remote_ip_prefix': None, u'id': u'c83371a8-f217-4038-982a-0165df8e9f4f'}], u'name': u'demosecuritygroup'}}
ae44389e-f884-4726-8bc3-ec3cbc7b876e

 

sbstep8

Step 8 – Create our security group rules with the following API call –

def create_security_group_rule(k5token, security_group_id, direction, portmin, portmax, protocol):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/security-group-rules')
    print networkURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.post(networkURL,
                                headers={'X-Auth-Token': token, 'Content-Type': 'application/json', 'Accept': 'application/json'},
                                json={
                                        "security_group_rule": {
                                            "direction": direction,
                                            "port_range_min": portmin,
                                            "ethertype": "IPv4",
                                            "port_range_max": portmax,
                                            "protocol": protocol,
                                            "security_group_id": security_group_id
                                            }
                                        })
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

again (repetitive I know) the call is invoked like so –

    # Create security group rules
    # allow rdp
    rdp_rule = create_security_group_rule(k5token, security_group_id, "ingress", "3389", "3389", "tcp")

    print rdp_rule

    print rdp_rule.json()

    # allow ssh # allow rdp
    ssh_rule = create_security_group_rule(k5token, security_group_id, "ingress", "22", "22", "tcp")

    print ssh_rule

    print ssh_rule.json()

     # allow icmp
    icmp_rule = create_security_group_rule(k5token, security_group_id, "ingress", "0", "0", "icmp")

    print icmp_rule

    print icmp_rule.json()

 

outputting –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/security-group-rules

{u'security_group_rule': {u'remote_group_id': None, u'direction': u'ingress', u'remote_ip_prefix': None, u'protocol': u'tcp', u'ethertype': u'IPv4', u'port_range_max': 3389, u'security_group_id': u'778052d3-d9b8-4eee-9ddc-655f185729ff', u'port_range_min': 3389, u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'id': u'7054f52e-b284-4f37-a859-e043dbb6393e'}}
https://networking.uk-1.cloud.global.fujitsu.com/v2.0/security-group-rules

{u'security_group_rule': {u'remote_group_id': None, u'direction': u'ingress', u'remote_ip_prefix': None, u'protocol': u'tcp', u'ethertype': u'IPv4', u'port_range_max': 22, u'security_group_id': u'778052d3-d9b8-4eee-9ddc-655f185729ff', u'port_range_min': 22, u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'id': u'f70815f8-ac23-43d9-b808-ebf8e12948a2'}}
https://networking.uk-1.cloud.global.fujitsu.com/v2.0/security-group-rules

{u'security_group_rule': {u'remote_group_id': None, u'direction': u'ingress', u'remote_ip_prefix': None, u'protocol': u'icmp', u'ethertype': u'IPv4', u'port_range_max': 0, u'security_group_id': u'778052d3-d9b8-4eee-9ddc-655f185729ff', u'port_range_min': 0, u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'id': u'49bdf8b6-618c-4714-b112-7420d815cd83'}}

 

sbstep9

Step 9 – Let’s grab a port from our new subnet that will be attached to the server with an api call like this –

def create_port(k5token, name, network_id, security_group_id, availability_zone):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/ports')
    print networkURL
    token = k5token.headers['X-Subject-Token']
    try:
        response = requests.post(networkURL,
                                 headers={
                                     'X-Auth-Token': token, 'Content-Type': 'application/json', 'Accept': 'application/json'},
                                 json={"port":
                                       {"network_id": network_id,
                                        "name": name,
                                        "admin_state_up": True,
                                        "availability_zone": availability_zone,
                                        "security_groups":
                                        [security_group_id]}})
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

invoked like so –

    # Create a new port for the server
    server_port = create_port(k5token, "demoserverport", network_id, security_group_id, "uk-1b")

    print server_port

    print server_port.json()

    server_port_id = server_port.json()['port'].get('id')

    print server_port_id

 

producing the following output –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/ports

{u'port': {u'status': u'DOWN', u'name': u'demoserverport', u'allowed_address_pairs': [], u'admin_state_up': True, u'network_id': u'bc491d0d-56c2-4706-b7b4-6e97029dbeaf', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'availability_zone': u'uk-1b', u'binding:vnic_type': u'normal', u'device_owner': u'', u'mac_address': u'fa:16:3e:60:d6:31', u'fixed_ips': [{u'subnet_id': u'bafdfc08-b55b-4a99-b1c9-2f43cf661a54', u'ip_address': u'192.168.10.2'}], u'id': u'f1db8672-2923-46fb-a65b-4f1e643f5a9e', u'security_groups': [u'7e35e0bc-6254-4ce6-9233-fce162db4a8e'], u'device_id': u''}}
f1db8672-2923-46fb-a65b-4f1e643f5a9e

 

sbstep10

Step 10 – We’re almost there. The last piece of the puzzle before we can deploy a server is the SSH key. This next function creates an SSH key for you – please be sure to store the Private Key somewhere safe. K5 only stores the public keys by design.

def create_keypair(k5token, keypair_name, availability_zone):

    computeURL = unicode(get_endpoint(k5token, "compute")) + unicode('/os-keypairs')
    print computeURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.post(computeURL,
                                headers={
                                     'X-Auth-Token': token,
                                     'Content-Type': 'application/json',
                                     'Accept': 'application/json'},
                                json={
                                    "keypair": {
                                        "name": keypair_name,
                                        "availability_zone": availability_zone
                                        }})
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

invoked this way –

    # Create ssh key pair that can be injected into the server
    server_key = create_keypair(k5token, "demokeypair", "uk-1b")

    print server_key

    print server_key.json()

    server_key_id = server_key.json()['keypair'].get('id')

    print server_key_id

    server_key_private = server_key.json()['keypair'].get('private_key')

    print server_key_private

    server_key_public = server_key.json()['keypair'].get('public_key')

    print server_key_public

 

should produce –

https://compute.uk-1.cloud.global.fujitsu.com/v2/7015d1478a4c4bd7b970215d7b0260dd/os-keypairs

{u'keypair': {u'public_key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu6R92WdwAgPJrWiwkd+wdHQ61ZX/1PL+yVWUXZ86Of+sHhHFUN2vCj6YfzuqKNJaAYjvy6Mga6rT1VBNyoD5EPryaop7w/1GMzRdZ7UaAwtJnNezu8bG054BVpTJ6vspNYsEGZs0KpggsN5O0f0qa2inxq/jDGTy+Le1qWBaQqB33PPVsH6y8a2DgZmDp6n8BY1DTd32GQzprq7ARfWhNu3nNXYmjAjUe/F1H2z2FmkfTDGvs1Un8ETUdywK7Deh938Nqi7agT7ECYSvawlAjwiBILjx0EsERQJr02dRw/BsgbeLsWY50X3l76tmRaiNE95jDGgzovGDBwm8fIL9LQ== Generated by Nova\n', u'private_key': u'-----BEGIN RSA PRIVATE KEY-----\nMIIEoQIBAAKCAQEAu6R92WdwAgPJrWiwkd+wdHQ61ZX/1PL+yVWUXZ86Of+sHhHF\nUN2vCj6YfzuqKNJaAYjvy6Mga6rT1VBNyoD5EPryaop7w/1GMzRdZ7UaAwtJnNez\nu8bG054BVpTJ6vspNYsEGZs0KpggsN5O0f0qa2inxq/jDGTy+Le1qWBaQqB33PPV\nsH6y8a2DgZmDp6n8BY1DTd32GQzprq7ARfWhNu3nNXYmjAjUe/F1H2z2FmkfTDGv\ns1Un8ETUdywK7Deh938Nqi7agT7ECYSvawlAjwiBILjx0EsERQJr02dRw/BsgbeL\nsWY50X3l76tmRaiNE95jDGgzovGDBwm8fIL9LQIBIwKCAQBAVad9vRB1uCfjrt1z\n16o2jkAOtxXinHSczOJpPegT4qFpZS21U1H04kLpy1BIgqKELvMhPz5QzNrqDOd4\noT9ziT0r17VnxI0Y3sDwW1lgISCNi8iYJuUVWr6hV5WvppHJNvoXaGmoNCh3J6X+\n2nTxoDmF8ylGE/uIeYAc0JP5lYQfHpMkMwG1pLOLf6Eb7OZg752+Dp07jedPabYx\n1i/PIkozmslKbgGM3R2CmhoYEWMKpmoOnPlzb4blRcxCiopSWQuhlj/2EI0sE1hc\n6+NE6pbQ5ddsZmm4DvQTACfy/l8H444DvLOAU2zEOVIp5hkCLP4Yv6YBBjfClqWq\nE7YLAoGBAOE/6AqprnrKzaAcboSiOMW4oidUKFFe9heqJJC54LIqGFyl1qEeJAvt\nllr9DjVA1xY/PRrCtb3D8y5ZJXmHjiRmwT/3UWW/Y0Zy+RkewLbmr0X/jVjk75Ns\n6wTcHlW/6/GEj9ny2ed+kul8IKOH1CFtxEVjiaTGXPy1yvxGSezlAoGBANVCR/My\nkMh/v4JjoWkeqOuyMDqPYFohuO9c6zF6uY5GBtzOcWT/eUQI1i66x/S6Cf2DOd+b\nVZKbobIBUFL36r915MEbkeTe1eHV/dSSmIgfV63qjTnvMLx/QR8VMzIp8+XQz/4z\npfFx96nh45MY9U1fwY3gr5VZl7kEXLJbf5KpAoGBAK3Dlb8VLtPBBQZ8VUG++JiH\nHgEVCSjUOi+DQMdq0eiG35ftpZI70qodroC0le6RGvPnpCqWNG3RrPfeXrzWSRTE\nS++ERhtKf8ih1hqxUtY87ZxmDfQeUmpp9yEBk75+HGnbSmZNoMiGNtFfwWg1lQPY\nVZSdPkvpeu7VYhMRpr4PAoGAc8TlOuDvgskBj+zxORCk17iAlNF9csHZa/892Qgb\nlmCHYexpcVd8WCIN3tpsjCp6c62jEvyNihn4sRan8oaVYJ8Vx+plJHj3w71PR4LH\n1N3QgvurwF1GV7LLlIfZ9qkPX3/74cREI/wKGmSuvZE0tPl3s2tYAJ5ZrZSnWYIg\nrrMCgYBvNCvqchbyDrClZTDJqAm+dpMKAdhKABHpuQUqG3hI+3M/hvjCRmFvLPdZ\nD1LNYXX2xB695iXuwF0Ekyvlf0QajToswSrl3wF06osObJ3iGb1IMtfTmb++JL9d\ncKclHW55JWix70Be6/krDsR6CSGT8Q1DiAJDxYxlzlb496AJ8A==\n-----END RSA PRIVATE KEY-----\n', u'user_id': u'03758af9df694db784c3547ab8217abb', u'name': u'demokeypair', u'fingerprint': u'38:a1:83:52:fd:fa:81:a3:87:3b:01:fc:0f:f1:8a:86'}}
None
-----BEGIN RSA PRIVATE KEY-----
MIIEoQIBAAKCAQEAu6R92WdwAgPJrWiwkd+wdHQ61ZX/1PL+yVWUXZ86Of+sHhHF
UN2vCj6YfzuqKNJaAYjvy6Mga6rT1VBNyoD5EPryaop7w/1GMzRdZ7UaAwtJnNez
u8bG054BVpTJ6vspNYsEGZs0KpggsN5O0f0qa2inxq/jDGTy+Le1qWBaQqB33PPV
sH6y8a2DgZmDp6n8BY1DTd32GQzprq7ARfWhNu3nNXYmjAjUe/F1H2z2FmkfTDGv
s1Un8ETUdywK7Deh938Nqi7agT7ECYSvawlAjwiBILjx0EsERQJr02dRw/BsgbeL
sWY50X3l76tmRaiNE95jDGgzovGDBwm8fIL9LQIBIwKCAQBAVad9vRB1uCfjrt1z
16o2jkAOtxXinHSczOJpPegT4qFpZS21U1H04kLpy1BIgqKELvMhPz5QzNrqDOd4
oT9ziT0r17VnxI0Y3sDwW1lgISCNi8iYJuUVWr6hV5WvppHJNvoXaGmoNCh3J6X+
2nTxoDmF8ylGE/uIeYAc0JP5lYQfHpMkMwG1pLOLf6Eb7OZg752+Dp07jedPabYx
1i/PIkozmslKbgGM3R2CmhoYEWMKpmoOnPlzb4blRcxCiopSWQuhlj/2EI0sE1hc
6+NE6pbQ5ddsZmm4DvQTACfy/l8H444DvLOAU2zEOVIp5hkCLP4Yv6YBBjfClqWq
E7YLAoGBAOE/6AqprnrKzaAcboSiOMW4oidUKFFe9heqJJC54LIqGFyl1qEeJAvt
llr9DjVA1xY/PRrCtb3D8y5ZJXmHjiRmwT/3UWW/Y0Zy+RkewLbmr0X/jVjk75Ns
6wTcHlW/6/GEj9ny2ed+kul8IKOH1CFtxEVjiaTGXPy1yvxGSezlAoGBANVCR/My
kMh/v4JjoWkeqOuyMDqPYFohuO9c6zF6uY5GBtzOcWT/eUQI1i66x/S6Cf2DOd+b
VZKbobIBUFL36r915MEbkeTe1eHV/dSSmIgfV63qjTnvMLx/QR8VMzIp8+XQz/4z
pfFx96nh45MY9U1fwY3gr5VZl7kEXLJbf5KpAoGBAK3Dlb8VLtPBBQZ8VUG++JiH
HgEVCSjUOi+DQMdq0eiG35ftpZI70qodroC0le6RGvPnpCqWNG3RrPfeXrzWSRTE
S++ERhtKf8ih1hqxUtY87ZxmDfQeUmpp9yEBk75+HGnbSmZNoMiGNtFfwWg1lQPY
VZSdPkvpeu7VYhMRpr4PAoGAc8TlOuDvgskBj+zxORCk17iAlNF9csHZa/892Qgb
lmCHYexpcVd8WCIN3tpsjCp6c62jEvyNihn4sRan8oaVYJ8Vx+plJHj3w71PR4LH
1N3QgvurwF1GV7LLlIfZ9qkPX3/74cREI/wKGmSuvZE0tPl3s2tYAJ5ZrZSnWYIg
rrMCgYBvNCvqchbyDrClZTDJqAm+dpMKAdhKABHpuQUqG3hI+3M/hvjCRmFvLPdZ
D1LNYXX2xB695iXuwF0Ekyvlf0QajToswSrl3wF06osObJ3iGb1IMtfTmb++JL9d
cKclHW55JWix70Be6/krDsR6CSGT8Q1DiAJDxYxlzlb496AJ8A==
-----END RSA PRIVATE KEY-----

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu6R92WdwAgPJrWiwkd+wdHQ61ZX/1PL+yVWUXZ86Of+sHhHFUN2vCj6YfzuqKNJaAYjvy6Mga6rT1VBNyoD5EPryaop7w/1GMzRdZ7UaAwtJnNezu8bG054BVpTJ6vspNYsEGZs0KpggsN5O0f0qa2inxq/jDGTy+Le1qWBaQqB33PPVsH6y8a2DgZmDp6n8BY1DTd32GQzprq7ARfWhNu3nNXYmjAjUe/F1H2z2FmkfTDGvs1Un8ETUdywK7Deh938Nqi7agT7ECYSvawlAjwiBILjx0EsERQJr02dRw/BsgbeLsWY50X3l76tmRaiNE95jDGgzovGDBwm8fIL9LQ== Generated by Nova

 

sbstep11

Step 11 – The one we’ve all been waiting for, deployment of the virtual machine. For this you’ll notice that I’ve configured two more pre-requisite parameters the id of the flavor or t-shirt size of the instance’s vCPU and RAM, and the id of the operating system image that we’ll use. The API call looks like this –

def create_server_with_port(k5token, name, imageid, flavorid, sshkey_name, security_group_name, availability_zone, volsize,  port_id):

    computeURL = unicode(get_endpoint(k5token, "compute")) + unicode('/servers')
    print computeURL
    token = k5token.headers['X-Subject-Token']
    try:
        response = requests.post(computeURL,
                                headers={'X-Auth-Token':token,'Content-Type': 'application/json','Accept':'application/json'},
                                json={"server": {

                                                 "name": name,
                                                 "security_groups":[{"name": security_group_name }],
                                                 "availability_zone":availability_zone,
                                                 "imageRef": imageid,
                                                 "flavorRef": flavorid,
                                                 "key_name": sshkey_name,
                                                 "block_device_mapping_v2": [{
                                                                               "uuid": imageid,
                                                                               "boot_index": "0",
                                                                               "device_name": "/dev/vda",
                                                                               "source_type": "image",
                                                                               "volume_size": volsize,
                                                                               "destination_type": "volume",
                                                                               "delete_on_termination": True
                                                                            }],
                                                 "networks": [{"port": port_id}],
                                                 "metadata": {"Example Custom Tag": "Finance Department"}
                                                }})

        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

and is called like this –

    # K5 predefined parameters for images and flavors

    # this is the id of the K5 ubuntu image
    image_id = "ffa17298-537d-40b2-a848-0a4d22b49df5"

    # the is the id of a small flavor size (T-1 or P1...need to verify)
    flavor_id = "1901"

    # Create the virtual machine
    new_server = create_server_with_port(k5token, "demoserver", image_id, flavor_id, server_key_name, security_group_name, "uk-1b", 3,  server_port_id)

    print new_server

    print new_server.json()

 

outputting the following –

https://compute.uk-1.cloud.global.fujitsu.com/v2/7015d1478a4c4bd7b970215d7b0260dd/servers

{u'server': {u'links': [{u'href': u'http://10.23.0.201/v2/7015d1478a4c4bd7b970215d7b0260dd/servers/5c57d8bc-6c2d-42bf-a84a-77745651b8e4', u'rel': u'self'}, {u'href': u'http://10.23.0.201/7015d1478a4c4bd7b970215d7b0260dd/servers/5c57d8bc-6c2d-42bf-a84a-77745651b8e4', u'rel': u'bookmark'}], u'OS-DCF:diskConfig': u'MANUAL', u'id': u'5c57d8bc-6c2d-42bf-a84a-77745651b8e4', u'security_groups': [{u'name': u'demosecuritygroup'}]}}

 

And that’s all there is to it! You should now have a new instance running on K5.
The next step is only necessary if you’d like to attach an external/public floating ip address to your new instance.

sbstep12

Step 12 – Add a floating or global ip address to the new virtual machine by using the following API call –

def create_global_ip(k5token, ext_network_id, port_id, availability_zone):

    networkURL = unicode(get_endpoint(k5token, "networking")) + unicode('/v2.0/floatingips')
    print networkURL
    token = k5token.headers['X-Subject-Token']

    try:
        response = requests.post(networkURL,
                                headers={
                                     'X-Auth-Token': token,
                                     'Content-Type': 'application/json',
                                     'Accept': 'application/json'},
                                json={
                                             "floatingip": {
                                                     "floating_network_id": ext_network_id,
                                                     "port_id": port_id,
                                                     "availability_zone": availability_zone
                                                     },
                                            })
        return response
    except:
        return ("\nUnexpected error:", sys.exc_info())

 

which should look like this when invoked –

    # Assign a global/public ip address
    public_ip = create_global_ip(k5token, extaz2, server_port_id, "uk-1b")

    print public_ip

    print public_ip.json()

    print public_ip.json()['floatingip'].get('floating_ip_address')

 

and output this –

https://networking.uk-1.cloud.global.fujitsu.com/v2.0/floatingips

{u'floatingip': {u'router_id': u'4dde5593-6657-4dea-9095-e0f80129e3f4', u'status': u'DOWN', u'availability_zone': u'uk-1b', u'tenant_id': u'7015d1478a4c4bd7b970215d7b0260dd', u'floating_network_id': u'd730db50-0e0c-4790-9972-1f6e2b8c4915', u'fixed_ip_address': u'192.168.10.2', u'floating_ip_address': u'62.60.42.67', u'port_id': u'f5b24292-8a73-4903-afe1-a71da61a18db', u'id': u'77214fa3-e04a-40f6-93a6-400743bb0862'}}
62.60.42.67

sbstep13

The above ip address can be used to access your server from the internet!

Please note that this is deliberately a quite detailed tutorial. It’s also possible to do this without as many steps i.e. not creating a port first. This is for users that require that little bit extra detail.

The full listing of the example tutorial is here..

or can be downloaded from github here..

https://github.com/allthingsclowd/OpenStack_Fujitsu_K5_Server_Build_API_Demo

Please ensure to configure the input parameters located in the main function to match your target environment. There’s also a purge script that I created in my previous post that is quite handy here for testing.

Happy Stacking!

#withk5youcan

 

One thought on “From Zero to Hero with the K5 OpenStack IaaS API (Python 2.7X)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s