Tag Archives: vRA

HOWTO: Speed up vRA template deployments

[ Since flowgrab is no longer amongst us you can get it from dropbox now ]

I recently did some scalability and load testing on a vRA deployment. One of the problems I encountered was that the template deployments,which seemed reasonably fast suddenly became a bottleneck. So obviously I created a workflow to fix that. So here is howto: speed up vRA template deployments.

TL;DR: if you’re too lazy to read an just want the workflow click here.

Let me explain the scenario first: Usually when you deploy a virtualmachine from a vSphere template it is reasonably quick. After all it just copies a file from disk to disk. If you happen to have an all flash array (my customer does) it is probably pretty fast.  Especially when your array is VAAI compatible. Because that would offload the whole copy action to the array.A typical VAAI accelerated template deployment takes around 4 seconds on an EMC X-IO.

But what happens if your template is in one cluster and needs to be deployed to another cluster which has a separate storage array. Then VAAI can’t  do anything for you and the template will be copied over the network. Which could be rather slow compared to a VAAI accelerated deployment. More in the region of 2 minutes instead of 4 seconds.

So before we get to the solution let me answer a few questions:

1: Why do you have the template on another cluster? Well… this customer has more than 10 different clusters with their own storage. Why that is is a completely different sotry. Anyhow, I wasn’t going to copy the template to each of them manually. Keeping them up to date would be a nightmare.  Second reason is that that strategy would require another blueprint for each cluster. Which is another nightmare to maintain.

2: Why don’t you use a template LUN shared between all clusters? This was actually not physically possible and also otherwise unwanted. It also wouldn’t fix the VAAI issue since it would require copying from one storage bo to another. It would be faster then copying over the network, that’s for sure.

3: 2 minutes isn’t that long. Why bother? Actually, consider the fact that vRA by default only does 2 deployments in parallel. That means that if you kick-off 50 deployments it takes at least 48 minutes before it even start deploying the last request. That is an unacceptable  long delay and even causes time-outs in vRA. I tried bumping up the number of parallel deployments but that slowed the deployments so much they never finished within vRA time-outs.

So…. I didn’t want a separate blueprint for each target cluster and I didn’t want to copy the template manually. The solution that remains is having the template copied by a workflow and then overwrite the template that the blueprint is going to use. Can we do that? You we can!  :). Turns out there is a custom propery called __clonefrom which contains the template name. If you overwrite this property during the buildingMachine state it will just use that machine to clone from.

to automate this process I created a workflow that:

  1. gets the template name from the __clonefrom property
  2. gets the cluster name of the cluster where vRA is going to deploy the machine
  3. add the cluster name to the tempate name and checks if such template exists
  4. If it doesn’t exist it will clone the template that is configured in the blueprint to the target cluster and adds the cluster name to the template name so we can find it next time we deploy something to the same cluster.
  5. overwrite the __cloneid property and then let vRA do it’s jobs

selectTemplate workflow

That’s it. This will make sure you have VAAI enabled deployments on each cluster. In my case it decreased the template deployment time from around 2 minutes to 4 seconds. This is so fast tha vsphere deployment is done before vRA can kick-off the next one.

you can download the workflow from dropbox. Use at you own risk! It’s tested wit vRA 6.2.1 and vSphere 5.5. Should work on any vRA 6.x version or even 5.x but I didn’t test that. Not sure about vRA 7. I’ll let you know when I had a change to test that.

vRA 7 Blueprint designer preview

One of the big changes you’ll see in vRA 7 is the new blueprint designer. If you can’t wait until the GA release, here is a vRA 7 bleuprint designer preview.

In summary these are the differences from the old pre-7 blueprints:

  • Graphical canvas
  • Unified blueprint. No difference between single machine, multi-machine or application blueprints
  • NSX integration. You can now drag and drop NSX object into your blueprint canvas
  • Ability to use blueprints in blueprints Yes, nested blueprints!
  • Ability to set dependencies between elements in the blueprint
  • Ability to use vRO workflow in a blueprint. This is now called XaaS, formerly ASD
  • Last but not least: The designer is on a separate tab in the vRA GUI not hidden in the infrastructure tab.

So basically all the good elements of application director (or appD or application services) are now integrated into the vRA blueprint designer. On top of that we can now use workflows directly in a blueprint. Which is exciting to me really.

This is what the designer looks like when you just created a new blueprint:


And this is what it looks like when you would create a typical two tier application:


Please note that these screenshots are taken from a BETA version so the GA release might look a little bit different.

As you can tell from the screenshots all features are now nicely integrated. You no longer need to hack NS integration into the product or use vRO to actually integrate app services and vRA.

I don’t want to complain because this is a huge step forward but…. The thing I’m still missing though is a higher abstraction layer on top of the blueprints. The bueprint author still has to decide one which reservation policy the VM will land for example. If you have different environments you’ll end up with blueprint sprawl. A better approach would be a form that asks functional questions like: is it for dev or prod? and then populates the different properties in the blueprint. I used to build that kind of functionality in workflows and then publish those in the catalog instead of the actual blueprints. Hopefully VMware will build something into the product to facilitate this.

VMworld news: vRealize Automation 7

I’m currently at VMworld Barcelona where this morning VMware announced the new version of vRA. It’s going to be called vRA7. As far as I know it’s still not GA (general availability)  but the Beta program is in full swing.

This morning I attended a a vExpert deepdive session hosted by @virtualJad. Here is an overview of some of the new features:

  • Simplified architecture You no longer need 24 machines for an enterprise deployment.
  • Simplified installation: From download to up and running in 20 minutes. All wizard driven so anybody can run a vRA PoC without assistance from PSO or other consultants.
  • One converged blueprint. No difference between IaaS and ASD (which is now called XaaS by the way) and application blueprint
  • Blueprint designer is now a beautiful canvas which allows for visio style drag and drop design.
  • Nested blueprints: You can use other blueprints in your blueprint (nice!)
  • Eventbroker: instead of having a few workflow stub we can now create policies that define when to kickoff a workflow. There are 60 different lifecycle events to which you can attach workflows. And each event has multiple stages (pre, event, after).

This event broker makes the product so much more extensible than what it currently is. The possibilities are almost endless. The other nice thing about this is that it is policy driven and defined by the vRA admin. So extensibility is now no longer part of the workflows. This means you can give the workflow designer to an application architect while still making sure that important IPAM or CMDB workflow is kicked off with each deployment. The application architect can consume XaaS workflows to extend his own blueprint.

In summary: really cool stuff, you’ll be reading lots more about it here in the coming months. I know, I know, I haven’t blogged in a while but I promise you’ll see some good vRA 7 stuff here on this blog!

Automating vRA IaaS Blueprint creation with vRO

In some situations it might be very convenient to be able to automatically create vRA (vCAC) Iaas blueprints. However, since the IaaS part of vRA uses an OData API doing so is not a trivial task. OData is basically just a representation of the IaaS database. So there is no logic in front of it and thus no way to tell the API: “Create a blueprint for this VM please”. Previsously I talked a bit about the vCAC API. In this post I’ll get more practical and explain the steps involved when automating vRA IaaS blurptint creation with vRO.


All objects in the IaaS OData API are called entities. It doesn’t matter if it is a blueprint, a host or a build profile, everything is an entity. So in order to create a new blueprint we have to create a new entity. Which is done with the following line of code:

the createModelEntiy method  needs a couple of input parameters. The first one is easy, it’s the id of the vCACServer. This is the IaaS server object in vRO not the vCACCAFE host. You can just configure this server as an input of the workflow.

The second parameter is always “ManagementModelEntities.svc” and the third one is basically the table name of where the entity should end up. In the case of a blueprint entity it’s always “VirtualMachineTemplates” because that’s the table where the blueprints live.

Now for the tricky part: the parameter and links values.


The parameters for the entity are basically the properties of the object. The trick is figuring out which properties to use. There are a couple of methods. You could create a simple workflow that just dumps all the properties of an existing blueprint entity in a log. I prefer using LINQpad because it also shows you the relations between different tables.

So which properties do we need for the blueprint entity and how do we put them into the parameters variable? The script below shows how to do this.

Obviously you might want to change the TenantID or get it from a variable.


An entity object is basically an entry in a database table. This table links to other tables. So when creating a new entity you need to define to which elemetns in other tables this entity links to.

For blueprint entities there are 5 links to be set:

  • InterfaceType (vSphere in this case)
  • HostReservationPolicy (The reservation policy to deploy to)
  • ProvisioningGroup (A.ka Businiss Group)
  • WorkflowInfo (This is the kind of deployment you want. So for a BP that clones a template you need the Clone Workflow WorkflowInfo entity… still with me?)
  • GlobalProfiles (A.k.a Build Profiles)

This is how you put those links into a links object:

As you can see each attribute in the object is an array. The content of the array are vCAC Entities.  Most arrays have one value only the globalProfiles has multiple values if you have more then one build profile selected. In the code above I the buildProfiles variable is defined somewhere else and it is already an array so I left out the [ ].

Finding links entities

I guess you’re now wondering how to get the entities object for the links. You need to use the vCACEntitieManager to find these entities. Here is an example of how to find the WorkflowInfo entity for the clone workflow:

You can find the reservation policy by name in a very similar way:

If you have an array with build profile names as in input you can find all the corresponding entities with this piece of code:

If you want more of this in a workflow that’s ready to run, see the link at the end of this post.


So now we are able to create an entitie with the right parameters and links. After that’s successfully done there is one more thing we need to do: configuring the custom properties on the blueprint.

There are a couple required properties which start with a double underscore. Here is the list I used:

__buildprofile_order (order in which the buildprofiles are applied)
__clonefromid (id of the virtualmachineTemplate entity we created)
__clonefrom (name of the vmtemplate entity we created)
__clonespec (name of the customization spec in vCenter)
__displayLocationToUser (false)
__menusecurity_snapshotmanagement (false)
VirtualMachine.DiskN.IsClone (true for each disk)
VirtualMachine.DiskN.Size (the size of each disk)

You can set these properties with the “Update Property to Blueprint” workflow that comes with the vCAC plugin.

I used this script to generate the buildprofile order:

 Just give me the workflow

Too much information? I uploaded the example workflows to flowgrab. Find the download here.

A word of warning: The workflows might assume you’re using the vsphere.local tenant in vCAC. Should be easily fixable. If I’ve got some timeleft in the near future I might fix that myself. If you’re using another tenant you should be able to easily makes this work for you anyways.

Automating vRA (vCAC) using vRO – Split Brain

Recently I have done some work on automating vRA (vCAC) using vRO (vCO). This meant I had to dive into the vCAC APIs. The bad news is that this felt like diving into a pool of dark muddy water. The good news is that I’m still alive, my headache is gone and I’ll try to capture some of the things I learned on this blog.

Split brain

In this post I’ll start out with an introduction to the vCAC APIs. Yes, plural. Not just one API.

vCAC ahem… VRA is actually not just one product, it’s two products which are loosely coupled and sold as one. The first product is the vRA Appliance also known as CAFE. This is a new product that was introduced with vCAC verion 6.0. It is developed in Java (springsource), runs on linux, uses postgres as a data persistence layer, seems to use a micro services architecture , supports multi-tenancy and provides a REST API.

But there also is the old product that was originally developed at CreditSuise, spun off as DynamicOps and then acquired by VMware. It was sold as vCAC 5.x, is developed in .net, uses an MS SQL back-end, runs .net workflows, has no notion of multi-tenancy and provides an OData API. This part is usually called the Iaas Part.

The two products are also reflected in two separate vCO ahem… vRO Plugins. Although you download and install just one package there are really two plugins installed. One is called VCAC and has the description “vCloud Automation Center Infrastructure Administration plug-in for vCenter Orchestrator” the other one is called CAFE and is described as “vCloud Automation Center plug-in for vCenter Orchestrator”.

Confusing. Right? So let’s clear things up:

CAFE is the virtual appliance. All new features are developed in CAFE. So anything that was added since 6.0 runs on the appliance and can be used from the REST API. On top of that some functionality was moved to the appliance. Functionality running in CAFE in version 6.1 includes:

  • Business Groups and Tenants
  • Advanced Service Designer
  • The Catalog
  • Resource Actions
  • Approval policies
  • Notifications

So if you want to automate anything regarding any of these features you’ll need the CAFE plugin which talks to the REST API running on the virtual appliance.

IaaS is the name of everything that’s not on the appliance. It is the reason you need a windows server to run vRA, not just the appliance. This windows server (or multiple servers) runs the old DynamicOps Software with some modifications. Features provided by this part of vRA include:

  • Virtual Machine Blueprints
  • Machine Prefixes
  • Provisioning Groups (Maps to Business Groups in CAFE, GUI only knows Business Groups in the current version)
  • Reservations
  • VirtualMachines (vCAC VM objects which map to vSphere/vCloud VMs or even physical machines)

If you want to automate any of the above you’ll need to use the vCAC plugin or the Odata API. If you’re note familiar with Odata APIs there is something you should know: It’s not an actual API. It’s just a representation of the database. There is no application logic behind it, just database constraints. This means that creating new things (called entities) is rather difficult. You have to figure out all the links between different database tables yourself. I’ll try to dive into this deeper in another blog post.

There another peculiarity I want to point out: there is no multi-tenency in the IaaS part. This means that a lot of items from the IaaS part (for example: machine prefixes) are shown to all tenants!


The fact that vRA basically has a split brain provides some challenges when automation things in vRA. For Example: You’ll have to create a blueprint in the IaaS part but when you want to publish it you have to create a catalog item in the CAFE part of the product. Which brings me to the last part of this post.

As I said before the two product are loosely coupled. The actual touchpoints are not documented. Or at least I couldn’t find anything. But after spending a lot of hour trying to find out how to autmate the publishing of blueprint I found these touchpoints between both APIs:

  • The Business Group ID in CAFE is identical to the Provisioning Group ID in IaaS. If you create a Business Group in the REST API then vRA also creates the ProvisioningGroup in IaaS for you.
  • The catalog actually consists of three catalogs. More on this later. One of the catalogs is the provider catalog. Each provider manages its own provider catalog. IaaS is on of the providers. Somehow CAFE knows where to find certain provides IDs. Not sure where to find or set that mapping.
  • Every Catalog Item has a providerBinding attribute. This contains the bindingId. This binding ID is the blueprint ID (virtualMachineTemplateID) from the IaaS Part. This is how vRA figures out which blueprint to deploy when you request a catalog Item.
  • A Resource Operation has bindingId which maps the CAFE action to the IaaS action (like powerOn a VM for example)

Running vRA 6.2 without Windows

After the release of vRA 6.2 I wanted to setup a small lab on my laptop. I really didn’t feel like creating a windows machine and getting SQL and AD up and running. So I tried running vRA 6.2 without windows.

If you leave windows out of the vRA installation you’ll still end up with a useable vRA environment. I’ll go into what’s working and what’s not in a minute. Let me explain how I build the environment first.

Building the Windows less vRA environment

Below is a brief description of the deployment of the vRA components I used. I left DNS out of this because I just used local hosts files. That works fine for a lab. Obviously I don’t recommend any of this in a production scenario.

The first thing you’ll need is an authentication source. In every installation I have seen this is Microsoft Active Directory. But vRA also supports open ldap so that’s the obvious way around AD. I used the turnkey openldap appliance. After download, just open it in VMware workstation, start it and answer the few questions the setup asks you. Then open a browser and go to the IP address for the ldap vm. After logging in your screen should look something like this:


To create a new user account navigate to the users ou and click “Create new entry here”, then select “Generic: User account” and fill out the form. If you’re really lazy you can just use the existing admin user as a user in vRA. I also tried using the LDAP groups in vRA but somehow that doesn’t work for me so I’m just using a single user account from the LDAP.

After the ldap is setup you need to deploy the vRA identity appliance. After it’s deployed just start it, answer the questions and then go to the configuration page. configure the identity appliance as you would normally do, just don’t configure any AD stuff.  Once the identity appliance is up and running you deploy and configure the vRA (vCAC) appliance. No special configuration here.

When everything is installed and configured you should be able to connect to you vRA instance through the url: https://<vra appliance ip>/vcac. Login using the administrator@vsphere.local account. On the tenant tab click “Add Tenant”.


give the tenant a name and url and click “Submit and next”.


Now click “Add Identity store”


Connect to your ldap as shown above. The IP address is the IP address of the open LDAP appliance we deployed earlier.

After the Identity store is configured the next step is to add a Tenant admin.


As you can see in the screenshot you’ll be unable to add any infrastructure administrators. This is because all IaaS functionality in vRA is delivered through the IaaS windows components which we did not install.

Now logout and go to the tenant url for the tenant you just created: https://<vra host>/vcac/org/<tenant url>. You should be able to login with the tenant administrator account you configured in the previous step.


What doesn’t work?

  • As I wrote above, LDAP groups don’t seem to work. I don’t know if this is a vRA issue or an issue with the turnkey openLDAP appliance. I’ll need to investigate this further.
  • Any IaaS feature. This includes:
    • Machine Blueprints
    • Machine Reclemation
    • Business Groups (although they are in the CAFE API but definitely not in the GUI)
    • Fabric Groups
    • Reservations / Reservation Policies
    • Actually the whole “Infrastructure” tab is not available.


What works?

  • All ASD features
  • The catalog
  • Approval policies
  • Tenants
  • Branding



Setting up vRA without any windows components is really easy. Mostly because you can skip all the IIS, vCAC manager, DEM and other stuff that comes with it. It also doesn’t require a lot of resources. For me it runs fine a my laptop (i7, 16GB RAM, SSD). You are only able to use the ASD part of vRA but that is actually pretty powerfull. You’ll just have to do without the IaaS stuff. That doesn’t mean you can’t deploy any virtual machines, you can actually deploy them using an ASD workflow.

By the way: I have a feeling the performance of vRA is a lot better without the IaaS parts. But I have no numbers to support this.

vRealize 6.2 releases

VMware released version 6.2 of a lot of the vRealize familiy products, vRA is one of them. I’m not going to cover all new features here, you can find them in the release notes. One feature I really like is this one:

  • Ability to edit custom properties for published applications in the service catalog.

This feature means we can now have user input for IaaS Custom properties on Application blueprint catalog items. I used to create some fairly complicated ASD workflows to work around this so this new feature makes life a lot easier for those customers using Application Services.

A couple other features worth mentioning or those:

  • Schedule date and time for the reconfigure operation.
  • Supports configurable email templates.

VMware also released a new product: vRealize Code Stream. This is VMwares take on a Continuous delivery tool. Its is connecting existing tools an repositories and provides the use the ability to create pipelines containing tasks. Interaction with external systems is handled by vRO (former vCO).



VMworld and the future of Orchestrator

Last week I attended VMworld Europe in Barcelona. I had a great time, eating tapas, drinking Rioja and learning something new in between. I already wrote about elasticity achieved using project fargo and docker on my company blog. Since this blog is more automation focussed I wanted to highlight some automation news. OR actually it is more about the future of Orchestrator.

The first thing that stood out to me was the lack of vCenter Orchestrator uuhh vRealize Orchestrator break-out sessions. I think there were two or three session explicitly about Orchestrator. A couple others went a little bit into orchestrator but were focused on vRealize Automation (vCAC). Last year there were quite a couple of sessions about Orchestrator. Telling us it was the best kept secret or the best VMware product never released and we should really go and use this awesome tool. And of course they were right to say so. And seeing where VMware is going with Orchestrator I was really surprised they didn’t give it more attention during Vmworld.

Which brings me to my second point. It is clear by now that Orchestrator will be used as the back-end for vRealize Automation. We can already see this in the current versions: The integration with NSX is completely implemented using Orchestrator. vCAC ugh… vRA has no interaction with NSX whatsoever, everything is handled via Orchestrator.

The same goes for what VMware calls Anything as a Service. Which is delivered using the Advanced Services Designer. Yeah that’s a lot of buzzwords in one sentence. In reality it is just a forms designer which you can use to design user front-ends for Orchestrator workflows. The objects created by the workflow can then be managed by vRealize Automation.

I already see that the adoption of Orchestrator is mainly driven by the use of vCAC. But there is more to come. VMware told in one of the Orchestrator sessions that Orchestrator will be used as a DEM replacement for vRealize Automation (but any information given in such presentation may change at any time). For who isn’t familiar with vCAC/vRA; The DEM is the Distributed Execution Manager. It is basically the component which does the actual work in a vCAC deployment. Currently it is 100% .net code and runs MS .NET workflow foundation workflows. So it makes total sense to replace that workflow engine with VMwares own workflow engine. The result will be that some day we can get rid of the windows components in vCAC and end up with just a virtual appliance which is easy to deploy and configure. That day will be a good day.

To be able to use orchestrator on the scale that vRA requires there will be some changes to the product in the future. For example, better permission management, multi geographical deployment models, integration with DevOps solutions and a lot more.

So although Orchestrator didn’t get a lot of attention during VMworld it seems it is going to play a crucial role in VMware’s automation strategy. Nice 🙂