Category Archives: vRealize Automation (vCAC)

Automating vRA IaaS Blueprint creation with vRO

In some situations it might be very convenient to be able to automatically create vRA (vCAC) Iaas blueprints. However, since the IaaS part of vRA uses an OData API doing so is not a trivial task. OData is basically just a representation of the IaaS database. So there is no logic in front of it and thus no way to tell the API: “Create a blueprint for this VM please”. Previsously I talked a bit about the vCAC API. In this post I’ll get more practical and explain the steps involved when automating vRA IaaS blurptint creation with vRO.

Entities

All objects in the IaaS OData API are called entities. It doesn’t matter if it is a blueprint, a host or a build profile, everything is an entity. So in order to create a new blueprint we have to create a new entity. Which is done with the following line of code:

the createModelEntiy method  needs a couple of input parameters. The first one is easy, it’s the id of the vCACServer. This is the IaaS server object in vRO not the vCACCAFE host. You can just configure this server as an input of the workflow.

The second parameter is always “ManagementModelEntities.svc” and the third one is basically the table name of where the entity should end up. In the case of a blueprint entity it’s always “VirtualMachineTemplates” because that’s the table where the blueprints live.

Now for the tricky part: the parameter and links values.

Parameters

The parameters for the entity are basically the properties of the object. The trick is figuring out which properties to use. There are a couple of methods. You could create a simple workflow that just dumps all the properties of an existing blueprint entity in a log. I prefer using LINQpad because it also shows you the relations between different tables.

So which properties do we need for the blueprint entity and how do we put them into the parameters variable? The script below shows how to do this.

Obviously you might want to change the TenantID or get it from a variable.

Links

An entity object is basically an entry in a database table. This table links to other tables. So when creating a new entity you need to define to which elemetns in other tables this entity links to.

For blueprint entities there are 5 links to be set:

  • InterfaceType (vSphere in this case)
  • HostReservationPolicy (The reservation policy to deploy to)
  • ProvisioningGroup (A.ka Businiss Group)
  • WorkflowInfo (This is the kind of deployment you want. So for a BP that clones a template you need the Clone Workflow WorkflowInfo entity… still with me?)
  • GlobalProfiles (A.k.a Build Profiles)

This is how you put those links into a links object:

As you can see each attribute in the object is an array. The content of the array are vCAC Entities.  Most arrays have one value only the globalProfiles has multiple values if you have more then one build profile selected. In the code above I the buildProfiles variable is defined somewhere else and it is already an array so I left out the [ ].

Finding links entities

I guess you’re now wondering how to get the entities object for the links. You need to use the vCACEntitieManager to find these entities. Here is an example of how to find the WorkflowInfo entity for the clone workflow:

You can find the reservation policy by name in a very similar way:

If you have an array with build profile names as in input you can find all the corresponding entities with this piece of code:

If you want more of this in a workflow that’s ready to run, see the link at the end of this post.

Properties

So now we are able to create an entitie with the right parameters and links. After that’s successfully done there is one more thing we need to do: configuring the custom properties on the blueprint.

There are a couple required properties which start with a double underscore. Here is the list I used:

__buildprofile_order (order in which the buildprofiles are applied)
__clonefromid (id of the virtualmachineTemplate entity we created)
__clonefrom (name of the vmtemplate entity we created)
__clonespec (name of the customization spec in vCenter)
__displayLocationToUser (false)
__menusecurity_snapshotmanagement (false)
VirtualMachine.DiskN.IsClone (true for each disk)
VirtualMachine.DiskN.Size (the size of each disk)

You can set these properties with the “Update Property to Blueprint” workflow that comes with the vCAC plugin.

I used this script to generate the buildprofile order:

 Just give me the workflow

Too much information? I uploaded the example workflows to flowgrab. Find the download here.

A word of warning: The workflows might assume you’re using the vsphere.local tenant in vCAC. Should be easily fixable. If I’ve got some timeleft in the near future I might fix that myself. If you’re using another tenant you should be able to easily makes this work for you anyways.

Automating vRA (vCAC) using vRO – Split Brain

Recently I have done some work on automating vRA (vCAC) using vRO (vCO). This meant I had to dive into the vCAC APIs. The bad news is that this felt like diving into a pool of dark muddy water. The good news is that I’m still alive, my headache is gone and I’ll try to capture some of the things I learned on this blog.

Split brain

In this post I’ll start out with an introduction to the vCAC APIs. Yes, plural. Not just one API.

vCAC ahem… VRA is actually not just one product, it’s two products which are loosely coupled and sold as one. The first product is the vRA Appliance also known as CAFE. This is a new product that was introduced with vCAC verion 6.0. It is developed in Java (springsource), runs on linux, uses postgres as a data persistence layer, seems to use a micro services architecture , supports multi-tenancy and provides a REST API.

But there also is the old product that was originally developed at CreditSuise, spun off as DynamicOps and then acquired by VMware. It was sold as vCAC 5.x, is developed in .net, uses an MS SQL back-end, runs .net workflows, has no notion of multi-tenancy and provides an OData API. This part is usually called the Iaas Part.

The two products are also reflected in two separate vCO ahem… vRO Plugins. Although you download and install just one package there are really two plugins installed. One is called VCAC and has the description “vCloud Automation Center Infrastructure Administration plug-in for vCenter Orchestrator” the other one is called CAFE and is described as “vCloud Automation Center plug-in for vCenter Orchestrator”.

Confusing. Right? So let’s clear things up:

CAFE is the virtual appliance. All new features are developed in CAFE. So anything that was added since 6.0 runs on the appliance and can be used from the REST API. On top of that some functionality was moved to the appliance. Functionality running in CAFE in version 6.1 includes:

  • Business Groups and Tenants
  • Advanced Service Designer
  • The Catalog
  • Resource Actions
  • Approval policies
  • Notifications

So if you want to automate anything regarding any of these features you’ll need the CAFE plugin which talks to the REST API running on the virtual appliance.

IaaS is the name of everything that’s not on the appliance. It is the reason you need a windows server to run vRA, not just the appliance. This windows server (or multiple servers) runs the old DynamicOps Software with some modifications. Features provided by this part of vRA include:

  • Virtual Machine Blueprints
  • Machine Prefixes
  • Provisioning Groups (Maps to Business Groups in CAFE, GUI only knows Business Groups in the current version)
  • Reservations
  • VirtualMachines (vCAC VM objects which map to vSphere/vCloud VMs or even physical machines)

If you want to automate any of the above you’ll need to use the vCAC plugin or the Odata API. If you’re note familiar with Odata APIs there is something you should know: It’s not an actual API. It’s just a representation of the database. There is no application logic behind it, just database constraints. This means that creating new things (called entities) is rather difficult. You have to figure out all the links between different database tables yourself. I’ll try to dive into this deeper in another blog post.

There another peculiarity I want to point out: there is no multi-tenency in the IaaS part. This means that a lot of items from the IaaS part (for example: machine prefixes) are shown to all tenants!

Touchpoints

The fact that vRA basically has a split brain provides some challenges when automation things in vRA. For Example: You’ll have to create a blueprint in the IaaS part but when you want to publish it you have to create a catalog item in the CAFE part of the product. Which brings me to the last part of this post.

As I said before the two product are loosely coupled. The actual touchpoints are not documented. Or at least I couldn’t find anything. But after spending a lot of hour trying to find out how to autmate the publishing of blueprint I found these touchpoints between both APIs:

  • The Business Group ID in CAFE is identical to the Provisioning Group ID in IaaS. If you create a Business Group in the REST API then vRA also creates the ProvisioningGroup in IaaS for you.
  • The catalog actually consists of three catalogs. More on this later. One of the catalogs is the provider catalog. Each provider manages its own provider catalog. IaaS is on of the providers. Somehow CAFE knows where to find certain provides IDs. Not sure where to find or set that mapping.
  • Every Catalog Item has a providerBinding attribute. This contains the bindingId. This binding ID is the blueprint ID (virtualMachineTemplateID) from the IaaS Part. This is how vRA figures out which blueprint to deploy when you request a catalog Item.
  • A Resource Operation has bindingId which maps the CAFE action to the IaaS action (like powerOn a VM for example)

Running vRA 6.2 without Windows

After the release of vRA 6.2 I wanted to setup a small lab on my laptop. I really didn’t feel like creating a windows machine and getting SQL and AD up and running. So I tried running vRA 6.2 without windows.

If you leave windows out of the vRA installation you’ll still end up with a useable vRA environment. I’ll go into what’s working and what’s not in a minute. Let me explain how I build the environment first.

Building the Windows less vRA environment

Below is a brief description of the deployment of the vRA components I used. I left DNS out of this because I just used local hosts files. That works fine for a lab. Obviously I don’t recommend any of this in a production scenario.

The first thing you’ll need is an authentication source. In every installation I have seen this is Microsoft Active Directory. But vRA also supports open ldap so that’s the obvious way around AD. I used the turnkey openldap appliance. After download, just open it in VMware workstation, start it and answer the few questions the setup asks you. Then open a browser and go to the IP address for the ldap vm. After logging in your screen should look something like this:

openldap

To create a new user account navigate to the users ou and click “Create new entry here”, then select “Generic: User account” and fill out the form. If you’re really lazy you can just use the existing admin user as a user in vRA. I also tried using the LDAP groups in vRA but somehow that doesn’t work for me so I’m just using a single user account from the LDAP.

After the ldap is setup you need to deploy the vRA identity appliance. After it’s deployed just start it, answer the questions and then go to the configuration page. configure the identity appliance as you would normally do, just don’t configure any AD stuff.  Once the identity appliance is up and running you deploy and configure the vRA (vCAC) appliance. No special configuration here.

When everything is installed and configured you should be able to connect to you vRA instance through the url: https://<vra appliance ip>/vcac. Login using the administrator@vsphere.local account. On the tenant tab click “Add Tenant”.

add-tenant-gen

give the tenant a name and url and click “Submit and next”.

add-tenant-idstores

Now click “Add Identity store”

add-id-store

Connect to your ldap as shown above. The IP address is the IP address of the open LDAP appliance we deployed earlier.

After the Identity store is configured the next step is to add a Tenant admin.

tenant-admins

As you can see in the screenshot you’ll be unable to add any infrastructure administrators. This is because all IaaS functionality in vRA is delivered through the IaaS windows components which we did not install.

Now logout and go to the tenant url for the tenant you just created: https://<vra host>/vcac/org/<tenant url>. You should be able to login with the tenant administrator account you configured in the previous step.

 

What doesn’t work?

  • As I wrote above, LDAP groups don’t seem to work. I don’t know if this is a vRA issue or an issue with the turnkey openLDAP appliance. I’ll need to investigate this further.
  • Any IaaS feature. This includes:
    • Machine Blueprints
    • Machine Reclemation
    • Business Groups (although they are in the CAFE API but definitely not in the GUI)
    • Fabric Groups
    • Reservations / Reservation Policies
    • Actually the whole “Infrastructure” tab is not available.

tabs

What works?

  • All ASD features
  • The catalog
  • Approval policies
  • Tenants
  • Branding

adminscreen

Conclusion

Setting up vRA without any windows components is really easy. Mostly because you can skip all the IIS, vCAC manager, DEM and other stuff that comes with it. It also doesn’t require a lot of resources. For me it runs fine a my laptop (i7, 16GB RAM, SSD). You are only able to use the ASD part of vRA but that is actually pretty powerfull. You’ll just have to do without the IaaS stuff. That doesn’t mean you can’t deploy any virtual machines, you can actually deploy them using an ASD workflow.

By the way: I have a feeling the performance of vRA is a lot better without the IaaS parts. But I have no numbers to support this.

vRealize 6.2 releases

VMware released version 6.2 of a lot of the vRealize familiy products, vRA is one of them. I’m not going to cover all new features here, you can find them in the release notes. One feature I really like is this one:

  • Ability to edit custom properties for published applications in the service catalog.

This feature means we can now have user input for IaaS Custom properties on Application blueprint catalog items. I used to create some fairly complicated ASD workflows to work around this so this new feature makes life a lot easier for those customers using Application Services.

A couple other features worth mentioning or those:

  • Schedule date and time for the reconfigure operation.
  • Supports configurable email templates.

VMware also released a new product: vRealize Code Stream. This is VMwares take on a Continuous delivery tool. Its is connecting existing tools an repositories and provides the use the ability to create pipelines containing tasks. Interaction with external systems is handled by vRO (former vCO).

 

 

Prerequisites check vCAC 6.1 (automated)

As many of you already know VMware released vCloud Automation Center 6.1 September 9th. I’m working on an implementation already, so I feel pretty lucky. An installation always starts with the deployment of the necessary Windows (2012R2) servers and all the prerequisites listed in the documentation.

We at automate-it.today are big supporters in automating everything, as long as it useful or fun to build. So I was really excited about the pre-req automation script build by Brain Graf. It’s easy, it’s fast and will give you a good verification of all the necessary prerequisites.

You can find the script here.

 

vCAC 6.1 released

Today VMware released vCAC version 6.1. You can read all about the new features in the release notes.

vcac-splash

A few features stand out immediately:

  • Migrating a vCloud Automation Center version 5.2.1 or 5.2.2 deployment to version 6.1
  • Upgrading from vCloud Automation Center 6.0 to 6.1

Really VMware? The new feature is that we can migrate or upgrade to the new version? I can’t wait to see what will be in the next release… upgrading to 6.2 maybe?

Then there is this one:

  • Improvements in Reporting

That was actually there in 5.2 then removed in 6.0 but the database stored procedures were still there. So now they put back the widget in the interface which actually shows the reporting. I love it…..

But wait… there is more:

  • Application Services (Formerly VMware vCloud Application Director)

yep, a rename. Exciting.

All ranting aside. There are some noteworthy new features. These ones for example:

  • NSX for vSphere Integration

But: “All NSX for vSphere and vCloud Networking and Security integration is now through vCenter Orchestrator (vCO) plug-in, including previously released and new functionality.” So yeah… The free vCO plugin makes this all possible. Not the expensive vCAC product itself.

Ok, ok I’ll really stop ranting now. Because I really like the way this product is headed. VMware is leaving all the horrible .net stuff and moving all the executing onto vCO. And as you probably know I’m a huge vCO fan. So the relay good news is this:

  • Advanced Service Designer includes the following updates
    • State-Aware resource actions that allow a service architect to define how resource actions such as Discard, Provision, or Change, affect the lifecycle of a resource.
    • Any IaaS resource type can be extended with Advanced Service Designer resource actions.
    • Feedback is provided to the user on the state of their requests.

 

How to Automate: vCAC

This is the third blog in my “How to automate” series. In the first posts I dicussed PowerCLI and vCO. In this blog I will talk about vCAC.

vcac-splash

vCAC = VMware vCloud Automation Center. It was originally developed by Credit Suisse, then spun off into a company called Dynamic Ops before it was acquired by VMware. It is software that takes care of the buniness side of automation. It presents a portal on which users or admins can request machines and services. The logic inside vCAC determines who can request which services and where machines are deployed. After a VM or service is requested it  manages the lifecycle of the object. I use the word “object” on purpose here because it can be a virtual machine as well as a physical server. And since version 6 it can even be any kind of service instead of an actual server.

The Upsides

The big upside of vCAC is that it is a very versatile tool. Especially if you have the cloud developer license you can make the software do virtually anything. And even without it you are able to do a whole bunch of stuff. This includes deploying machines to both your internal cloud as well as public clouds like Amazon or vCloud powered clouds. You could also provision storage or networks if you need to. So the software was not only developed in Switserland, it’s also versatile like a Swiss army knife.

Another big plus in my opinion is that it integrates with vCenter ORchestrator so well. This is one of the reasons vCAC can do almost anything even without the developer license. If you can do it with vCO, you can integrate it with vCAC.

vCAC delivers an end user portal which doesn’t require you to do any web development. This makes building a portal very easy and leaves more time for you to focus on the automation that happens behind the portal.

vCAC also makes it possible to import existing virtual machines. This is something that is very hard with vCloud Director. This makes vCAC a much better fit than vCD for companies running their own private datacenter (or should I say “cloud”?). For public cloud providers vCD is still the way to go in my opinion although using vCAC is not impossible.

The Downsides

The fact that vCAC is so versatile is also a downside. vCAC needs a lot of configuration before it does what you want it to do. And that’s before you start digging in to custom workflows. There are 10 workflows which you can customize out of the box. If you want more you need the developer license. On top of that you need .NET knowledge to actually develop custom workflows.
And in depth .NET knowledge is not something you find in a regular vCloud admin.

But there is a way around this. That way is called vCO. You can call vCO workflows from the default vCAC workflows. This will solve most of your integration problems and I really like this approach.

Speaking about vCO integration, that is actually my next downside. Don’t get me wrong, I really like the vCO integration. It just leaves me wondering why I need a bunch of virtual machines, two database servers, an additional SSO and a lot of complexity to have a portal which basically calls vCO workflows and does some life cylce management. It just doesn’t feel right. Ok, vCAC can do a bit more then that. Deploying physical machines for example or creating datastores on NetApp storage. But most companies who will be using vCAC will probably not use these features. So they are stuck with an overcomplicated vCAC setup which has features that are already covered by vCO.

And that brings us to my last downside. The vCAC infra itself can get rather complex if you want to set it up in a redundant and scalable way. To VMware: Please simplfy this. I like the VA approach that was introduced with version 6 but I don’t like setting up 6 additional Windows machines and 5 loadbalancers.

The Right Tool for The Job?

When should you use vCAC? I my opinion you should use it when you regularly deploy new virtual machines or other services and you want to automate that process. Using vCAC forces you to automate every single step in the deployment process. This includes creating change requests, registering IP addresses, creating the VM, assigning a network and so on. This makes it possible for application administrators and developers to request machines without any intervention of a cloud administrator. And it doesn’t stop there. You can also automate the decommissioning process, limit the amount of resources someone can consume or allow them to deploy machines on Amazon instead of your internal cloud.

But remember: vCAC always brings a friend to the party: vCenter Orchestrator. But in my opinion that only makes the party better.

Selecting Network Through vCAC Property

vCAC is a really powerful automation tool, primarily giving users the opportunity to request their own applications without interaction of the IT Department. In this particular case I was configuring a blueprint to be used by developers.

Using vCAC to clone vCenter templates is really simple; users request a machine to use in the development or in the production environment. While I was testing the deployment of this blueprint the department responsible for Active Directory complained because my test machines, without the correct naming convention, appeared in the production AD.

The reservation used for this blueprint contained two networks, a development and a production network. Normally after deploying a vCenter template you edit the Virtual Machine hardware and select the correct network, before powering on the VM. In this case the requesting user does not have permission to connect to vCenter and to change this setting, and vCAC will just power on the VM. So we need another solution.

The solution is surprisingly not that difficult, you can use a custom property within vCAC. I will describe this in a few steps.

The first step is to find out which networks are available, this can be done by editing the reservation for this particular blueprint.

networkproperty-1

The next step is creating a new property definition within the property dictionary:

networkproperty-2

Name: VirtualMachine.Network0.Name
Display Name: Select Network
Control Type: DropDownList
Required: Yes

After you created the definition you can edit the property attributes

networkproperty-3

Type: ValueList
Name: Network
Value: Add all choices comma separated.

Now when the user will request a blueprint, there is a required choice with the name Network and the choices you added, in this case Development-1 and Production-1.

networkproperty-4

vCAC Blueprint Configuration

Below is the vCAC configuration workflow about configuring a Blueprint. This blogpost is the fifth in the vCAC configuration series.

The action blocks are actually clickable and will show you the matching parts of the VMware documentation in a popup window.

Go back to the configuration steps overview.

 

A couple of interesting vCAC documentation links about Blueprints:

 

vCAC Configuration – Reservation

This blogpost is the fourth in the vCAC configuration series.

Before a user can request a machine there need to be available resources, this resources are created with the Fabric groups. Within this fabric you can create a reservation.

A fabric administrator creates a reservation to allocate provisioning resources in the fabric group to a specific business group.

A virtual reservation allocates a share of the memory, CPU and storage resources on a particular compute resource for a business group to use.

A physical reservation is a set of physical machines reserved for a business group to use. Unprovisioned physical machines must be added to a physical reservation before being provisioned or imported, and cannot be removed until they are decommissioned and become unprovisioned.

A cloud reservation provides access to the provisioning services of a cloud service account, for Amazon AWS, or to a virtual datacenter, for vCloud Director, for a business group to use.

A business group can have multiple reservations on the same compute resource or different compute resources, or any number of physical reservations containing any number of physical machines.

A compute resource can also have multiple reservations for multiple business groups. In the case of virtual reservations, you can reserve more resources across several reservations than are physically present on the compute resource. For example, if a storage path has 100GB of storage available, a fabric administrator can create one reservation for 50GB of storage and another reservation using the same path for 60GB of storage. You can provision machines by using either reservation as long as sufficient resources are available on the storage host.

You can reserve physical machines only for a single business group. Because physical machines do not belong to fabric groups, all fabric administrators can manage all physical machines and reserve them for a particular business group.

Reservation

  • A reservation can only contain one policy.
  • A policy can be used on multiple reservations
  • Only one policy can be added to a blueprint

Go back to the configuration steps overview.