Category Archives: vRealize Orchestrator (vCO)

The right tool for the job

I work with vRealize Automation and vRealize Orchestrator on a daily basis. And I really enjoy doing so, especially the custom code development part. vRO gives a lot of flexibility and it’s not often that I’m unable to build what my customers need. Whetever the request I usually find a way to emply vRA and vRO in such a way that if fulfills the customers need. But more and more often do I wonder if we’re using the right tool for the job.

Today I presented a break-out session during the annual NLVMUG UserCon. In the presentation we emphasized the importance of using the right tool for the job. After all, you don’t drive a nail in the wall with a power drill. you can do so if you really want to but you’ll probably spent more time than needed putting up your new painting and likely destroy you power drill in the process. It’s similar in enterprise IT: You can use a customizable tool like vRA/vRO for nearly anything. But that doesn’t mean you should.

But if you can make it work anyway then why not? First of all: If you’re using a product to do something that it wasn’t originally intended to do you’ll spent a lot of time and money to make it do what you actually want.  But getting the product to do that is only the beginning. Now you need to maintain the product customizations. chances are something will break at the next product upgrade. So you postpone the upgrade, then postpone again and in the end the upgrade never happens because the risk is just too high.

Let me give an example: Let’s say you’re trying to deploy in-house developed code through different life cycle stages. You could argue that everything needs to run on a virtual machine so you start out by automating virtual machine deployment. You’ll probably use vRA or something similar to do that for you. After this first step you realize that the code does not run on an bare OS, you may need IIS or .NET or Java or a bunch of shared libraries. So you decide to automate the deployment of middleware software as well. But that still isn’t enough to run the code. You also need a database, a load balancer, an SSL certificate and last but not least: you need a way to deploy the code to your machines and configure the way it’s running.  Oh and of course all this needs to be triggered by the code repository and be completely self service. By the time you have implemented all this you’ll have written tons of custom installation scripts and integration workflows.

Automating code deployment can be tricky to say the least. And in my opinion all this difficulty stems from the fact that we’re starting with the VM as the unit of deployment. The actual unit of deployment is the code/application your developers are writing. By using the wrong data as input for the tool selection you ended up with the wrong tool.

Luckily there are tools designed for application deployment. One of them is called Cloud Foundry. If you use the Pivotal distribution you can set it up in a day or so. And then your developers can just run cf push and their code is running. In the cloud. Sounds a lot better than writing countless installation scripts and custom integrations doesn’t it? Also, the Cloud Foundry platform gives you loads of options you wouldn’t have out of the box with tools like vRA: auto-scaling, easy manual scaling, application health monitoring, service bindings, application statistics, centralized logging, custom logging endpoints and lots more.

There is one major “drawback” however: your applications need to be cloud native or 12factor apps. But you’ll have to transform your apps into cloud native apps at some point in the future anyways so why not start now?

 

Automated directory synchronization of the vRA Identity Manager

Disclaimer: The API documentation has not yet been released, therefor I would like to notice that this is currently an unsupported method of triggering a directory sync.

During a recent project the customer requested the functionality to create a new business group with just one click. This should be a function to onboard new teams into the vRA environment, including the creation of Reservations and Active Directory groups.

In vRA 6 this would not have been a problem at all, but starting at vRA 7 the Identity Manager was introduced. The Identity Manager, in short the connection from vRA to Active Directory (AD), synchronizes AD content on a specific schedule. This means that while specifying the different AD groups in the new Business Group, these will not be visible immediately but after a synchronization.

As the customer stated, it should be an automated process, a click on the button. Waiting for the synchronization to take place is not an option.. We are automating this, right?! Therefor my colleague Marco van Baggum (#vMBaggum blog) came up with the idea to automate the synchronization of the identity manager. In a shady corner Marco found the necessary API calls and off we go!

The first step is to create the a new HTTP-REST endpoint in vRO. Run the workflow “Add a REST host” located at Library / HTTP-REST / Configuration and use the following settings:

Name vRA
URL https://<vRA FQDN>/ e.g. https://itqlab-vra.itqlab.local/
Authentication NONE

* The other settings are dependent on how vRA is set-up and how vRO connects to it.

A new endpoint in the inventory should pop up at the HTTP-REST plugin. Now right click this endpoint and run the workflow to add the additional REST operations to it.

Name Get Directories
Method GET
URL template /SAAS/t/{tenant}/jersey/manager/api/connectormanagement/directoryconfigs

 

Name Get Directory Sync Executions
Method GET
URL template /SAAS/jersey/manager/api/connectormanagement/directoryconfigs/{directoryId}/syncexecutions

 

Name Invoke Directory Sync
Method POST
Content-type application/json
URL Template /SAAS/jersey/manager/api/connectormanagement/directoryconfigs/{directoryId}/syncprofile/sync

 

Name Login
Method POST
Content-type application/json
URL Template /SAAS/t/{tenant}/API/1.0/REST/auth/system/login

 

The images below show the configured operations in vRO

This slideshow requires JavaScript.

Now the endpoint and operations are created, import the workflow package attached to this post. (nl.itq.psi.vidm Workflows)

When the workflow package is imported, open the Configuration Elements tab and edit the Endpoints configuration element located under the ITQ folder. Select the correct HTTP-REST endpoint and REST-Operations, insert the correct username, password and tenant to connect to vRA. As a side-note, the used API calls can only be used with a vRA local account. Domain accounts will throw an “Invalid Credentials” error. Make sure that the user has rights to execute a Directory Sync in vRA.

Now go back to the workflow overview and expand ITQ / PSI / VIDM / Helpers. You should have the same overview as in the image below.

vRO Workflow structure

Now execute the “Synchronize active directory” workflow and the synchronization will start!

vRO Workflow execution
vRO Workflow execution

Please note that these workflows are not production ready yet and bugs may exist!

Download nl.itq.psi.vidm Workflows!

vRO Code – Finding VirtualMachines by Custom property

For the current project I’m involved in, I was asked to deliver a list of vRA deployed machines that have a Production status.

At first I have been writing a short piece of code that obtained all vRA managed machines and for each machine gathered the customer properties. Creating this workflow actually took less time than the execution itself as the environment has about 4200 managed objects. Next to the fact that this is time consuming to wait for, it will also generate a lot of load on the vRO service and the vRA IaaS API.

The developer in me felt like improving this and move the functionality to the vRA IaaS API, the API nevertheless has the custom properties linked to the virtual machine entity object. Eventually, after some research on ODATA queries and how to query for properties within linked entities, I was able to write the following ODATA filter:

Putting the filter and the vCAC IaaS plugin logic together will form the following script that can be used in either a workflow or an action:

To elaborate  a little bit on the code snippet above:

  • First the property and it’s value are being specified
  • The second step is to setup the filter with the property and value
  • The third step is to actually perform the call to vRA IaaS to return an array of vCAC:Entity based on the filter.
  • The last step in the code is to System.log() the names of the VirtualMachines that match the query.

When necessary to have vCAC:VirtualMachine objects instead of vCAC:entity objects change the last part of the code to:

 

Conclusion

Gathering virtualmachines based on specific properties can be a hassle using ODATA queries as in some cases it is not completely clear on how to structure the query. Eventually when the query is ready and working it shows to be much faster than creating a script to “hammer” the API for data.  The two screenshots below show the actually difference between the initial code and the improved code. The first screenshot is the original code, it errors out after 30 minutes of API calls. The second screenshot is a capture of the improved code, it runs for only 1 second to return the list of VirtualMachines matching the filter.

log get virtual machines by property and value error
First attempt ended up in an error returned by the vRA IaaS API after 30 minutes of performing API calls.

 

log get virtual machines by property and value
Second attempt with improved code. The runtime of the script is now only a matter of seconds.

vRO Code – Calculate CIDR notation

Recently I ran into situations where I needed to supply a network address in CIDR notation to external systems (infoblox and the oracle GRID installer in my case) from a vRealize Orchestrator workflow.

CIDR notation looks like this: 192.168.1.20/24. So instead of specifying the subnet mask using the 4 bytes separated by dots it just tells how many bits are used for the network number.

The thing is, all you can get from vRA is the regular subnet mask (255.255.255.0 in this example). Sometimes you can get away with a solution as simple as a few ifs or a switch/case but that’s not really the way to properly fix this.  I wanted to solve this once and for all so I wrote some JavaScript code for vRealize Orchestrator that calculates the number of bits in the mask and creates the CIDR network notation for you. Here it is:

  • Inputs for the script object or action:
    • gateway (String)
    • subnetMask (String)
  • Output:
    • cidr (String)

This script generates the network number but it can easily be adapted to return the ip address itself in CIDR notation.

 

 

vRO Code – Finding vRA IaaS entities using OData query

odata logo

As I’ve explained in an earlier blog the vRA API is split into two parts: CAFE and Iaas. This post is about the latter which still contains a lot of the vRA entities. When working with those entities you regularly need to find them first. One of the methods for finding vRA Iaas entities is using an odata query.

Continue reading

Free tool: vRO API Explorer

If you’re working with vRealize Orhcestrator you are probably spending a lot of time in the API explorer. I find myself using it all the time. But it has some flaws and the functionality is limited. Two of my colleagues recognized that and decided to build their own vRO API Explorer. And they didn’t stop there… it is now available online for everyboy to use: vroapi.com

Screenshot from 2016-03-15 12:03:29

Continue reading

Orchestration and your configuration data

As this is my first blog post at automate-it.today I would like to start off with something less-technical and have a monologue about the Orchestration and your configuration data. This first post will be the first in a series of four; the following post will be more technical and describing the possibilities, technical implementation and their pro’s and con’s.

First off, what is configuration/automation data? In short: the data that supports your  automation in terms of decision making and logic.

Continue reading

vRealize Orchestrator 7 Release

As you’re probably aware by now VMware released vRealize Automation 7 today. I already discussed vRA 7 before and there are a lot of blogs out there writing about it.

What I didn’t blog about before is that with vRA7 VMware also released vRealize Orchestrator 7.  Since vRO is the magic sauce that makes vRA work anyways I’m far more interested in the vRO release. So I quickly downloaded the ova (yep, no windows installer anymore), imported the ova into VMware workstation and connected to it. All in under 7 minutes :).

I read on another blog that there now is an HTML 5 client. But nope…. only the control center is HTML5.  The client used to develop workflows is still the same Java contraption. But it supports reconnecting now. Which is very nice because that means you don’t loose any work when a vRO server stops responding. Other than that I couldn’t find any improvements in the client. Still no decent IDE like environment to write your scripts in, no improvements to the api explorer as far as I can see. I’ll digg deeper in the coming week and let you know what I find.

Other new features:

  • Centralized Server administration.
  • Easy cluster configuration.
  • Easy workflow troubleshooting and runtime metrics.
  • Enhanced log monitoring, log persistency and added ability to export logs for a particular workflow run.
  • Direct correlation of system properties and workflow performance through the embedded JMX console integration.
  • Significant Orchestrator client improvements, including a workflow tagging UI, client reconnect options and enhanced search capabilities.

Automating vRA IaaS Blueprint creation with vRO

In some situations it might be very convenient to be able to automatically create vRA (vCAC) Iaas blueprints. However, since the IaaS part of vRA uses an OData API doing so is not a trivial task. OData is basically just a representation of the IaaS database. So there is no logic in front of it and thus no way to tell the API: “Create a blueprint for this VM please”. Previsously I talked a bit about the vCAC API. In this post I’ll get more practical and explain the steps involved when automating vRA IaaS blurptint creation with vRO.

Entities

All objects in the IaaS OData API are called entities. It doesn’t matter if it is a blueprint, a host or a build profile, everything is an entity. So in order to create a new blueprint we have to create a new entity. Which is done with the following line of code:

the createModelEntiy method  needs a couple of input parameters. The first one is easy, it’s the id of the vCACServer. This is the IaaS server object in vRO not the vCACCAFE host. You can just configure this server as an input of the workflow.

The second parameter is always “ManagementModelEntities.svc” and the third one is basically the table name of where the entity should end up. In the case of a blueprint entity it’s always “VirtualMachineTemplates” because that’s the table where the blueprints live.

Now for the tricky part: the parameter and links values.

Parameters

The parameters for the entity are basically the properties of the object. The trick is figuring out which properties to use. There are a couple of methods. You could create a simple workflow that just dumps all the properties of an existing blueprint entity in a log. I prefer using LINQpad because it also shows you the relations between different tables.

So which properties do we need for the blueprint entity and how do we put them into the parameters variable? The script below shows how to do this.

Obviously you might want to change the TenantID or get it from a variable.

Links

An entity object is basically an entry in a database table. This table links to other tables. So when creating a new entity you need to define to which elemetns in other tables this entity links to.

For blueprint entities there are 5 links to be set:

  • InterfaceType (vSphere in this case)
  • HostReservationPolicy (The reservation policy to deploy to)
  • ProvisioningGroup (A.ka Businiss Group)
  • WorkflowInfo (This is the kind of deployment you want. So for a BP that clones a template you need the Clone Workflow WorkflowInfo entity… still with me?)
  • GlobalProfiles (A.k.a Build Profiles)

This is how you put those links into a links object:

As you can see each attribute in the object is an array. The content of the array are vCAC Entities.  Most arrays have one value only the globalProfiles has multiple values if you have more then one build profile selected. In the code above I the buildProfiles variable is defined somewhere else and it is already an array so I left out the [ ].

Finding links entities

I guess you’re now wondering how to get the entities object for the links. You need to use the vCACEntitieManager to find these entities. Here is an example of how to find the WorkflowInfo entity for the clone workflow:

You can find the reservation policy by name in a very similar way:

If you have an array with build profile names as in input you can find all the corresponding entities with this piece of code:

If you want more of this in a workflow that’s ready to run, see the link at the end of this post.

Properties

So now we are able to create an entitie with the right parameters and links. After that’s successfully done there is one more thing we need to do: configuring the custom properties on the blueprint.

There are a couple required properties which start with a double underscore. Here is the list I used:

__buildprofile_order (order in which the buildprofiles are applied)
__clonefromid (id of the virtualmachineTemplate entity we created)
__clonefrom (name of the vmtemplate entity we created)
__clonespec (name of the customization spec in vCenter)
__displayLocationToUser (false)
__menusecurity_snapshotmanagement (false)
VirtualMachine.DiskN.IsClone (true for each disk)
VirtualMachine.DiskN.Size (the size of each disk)

You can set these properties with the “Update Property to Blueprint” workflow that comes with the vCAC plugin.

I used this script to generate the buildprofile order:

 Just give me the workflow

Too much information? I uploaded the example workflows to flowgrab. Find the download here.

A word of warning: The workflows might assume you’re using the vsphere.local tenant in vCAC. Should be easily fixable. If I’ve got some timeleft in the near future I might fix that myself. If you’re using another tenant you should be able to easily makes this work for you anyways.