WebCommander Walnut Installation Walk-through

In a previous blog post from a far away history, I wrote about the WebCommander Fling (https://pascalswereld.nl/2013/10/30/webcommander/). Man that one is from 2013, I have been putting blog posts out there for a while now, hope you did find something useful on the blog…..
Anyhow back to this one. The WebCommander developer reached out in that previous post comment with a request to write-up a guide for WebCommander Walnut. I am writing it up as a walk-through to get it started and showing some output. If you would like some additions to the post, add some of the information your would like to see added to the post, or post questions / remarks and I will try to look if I can make some additions. But first….a little reminder about that commander out on the web…

What is WebCommander

WebCommander is a collection of web services around PowerShell and PowerCLI scripts. The interface can be used to provide users with scripts without them learning or knowing the PowerCLI commands. Or to give users access only to specific prepared tasks without giving them access to the web client (they still need to have permissions in the environment to do their operations). A great way in delegating specific tasks!

WebCommander was initially released and maintained as a VMware Fling. WebCommander was received very well by the community and saw the Fling being released as, and in turn moved to, an open source project on GitHub in 2014 (as announced on http://www.virtu-al.net/2014/09/03/webcommander-goes-open-source/).

The WebCommander project page can be found at: https://github.com/vmware/webcommander. This WebCommander version mainly uses XML with browser side transforming (XSLT). And when you hear version you know there might be another one, and yes there is WebCommander Walnut in a different branch.

WebCommander Walnut is to be used when :

  • you prefer JSON over XML,
  • combining commands in workflows for more or complex automation,
  • run local or cloud scripts (WebCommander Hybrid),
  • having a history,
  • 64-bit PowerShell,
  • more new features,
  • and a new User Interface

WebCommander

Take a look at WebCommander Walnut for yourself, go to GitHub: https://github.com/9whirls/webcommander_walnut

Installation Guide

Prepare the system:

Create a VM

Use Windows 2012R2 or Windows 2008R2 as the OS.

When using Windows2008R2 there are the following specifics:

  • Install .Net Framework 4.5.2. Needed for the installation of PowerShell v5 on 2008R2
  • Install PowerShell version 5

When using a fresh installation of Windows2012R2 install PowerShell Version 5.

For installation of the PowerShell version 5 install the Windows Management Framework 5.0 that can be downloaded as an update, or directly from https://www.microsoft.com/en-us/download/details.aspx?id=50395&ranMID=24542&ranEAID=TnL5HPStwNw&ranSiteID=TnL5HPStwNw-UGYM_0Jpr8QpSOcSBwTXfQ&tduid=(97816b302a22d507fcc1386696df4801)(256380)(2459594)(TnL5HPStwNw-UGYM_0Jpr8QpSOcSBwTXfQ)().

For Webcommander and PowerShell: Set-ExecutionPolicy Unrestricted -Force.

IIS Web-Server (including SubFeatures and Management Tools). Either use the Add Roles and Features GUI to install the Web Server role or use PowerShell:

Install-WindowsFeature Web-Server -IncludeManagementTools -IncludeAllSubFeature

PHP from https://php.iis.net. Click  ‘Install PHP now’ from the web site to download the latest version. Execute the downloaded exe to start the Web Platform Installer. Continue the installer with all the default options (you can change by clicking the options link) and accept to do the installation. The installer will download and install the prerequisites.

PHP IIS Installation

And click Finish when done.

Install MongoDB for commands history.

In short the procedure for MongoDB is:

MongoDB download CEIt should offer you the correct release and OS.

  • Install via the downloaded msi. Select complete or customize if you want. Complete will install in the default locations.
  • Add the installation location as a system path environment. The default installation location is C:\Program Files\MongoDB\Server\3.4\bin.
  • Use your powershell window used to install IIS or open a command prompt
  • MongoDB requires a data directory to store all data. MongoDB’s default data directory path is \data\db. Create this folder using the following a command line
md \data\db
  • Or use another location to suit your needs.
  • MongoDB also requires a location to store logs. Create the log folder using command line
md \data\log
  • Create a config file location with
md \data\conf
  • And add a text file mongodb.cfg there (watch the view – file extensions there!)
  • Add the following to the cfg file and save:
           systemLog:
                  destination: file
                  path: c:\data\log\mongod.log
           storage:
                  dbPath: c:\data\db

mongodb

  • Install MongoDB as a Windows service by running mongod.exe with –install parameter (as administrator!).
mongod.exe --config "C:\data\conf\mongodb.cfg" --install

If you get api-ms-win-crt-runtime-l1-1-0.dll is missing from your computer like this

System Error - Mongod

your Windows updates either screwed up or you have to install Visual C++ Redistributable. (Re)installing Visual C++ will mostly do the trick.

  • And now we will have a MongoDB service (use –serviceName and –serviceDisplayName to change to another name if you wish).
  • Start the MongoDB service with net start MongoDB.
  • Create database and collection in MongoDB for WebCommander by running the commands below:
    • exe
    • use webcmd
    • createCollection(“history”)
    • Mongo should respond with “ok”:1
  • Install the MongoDB powershell module:
    • In PowerShellv5
Install-Module Mdbc
    • Accept the installation of required components.

Install latest version of VMware PowerCLI (version 6.5.1 at time of writing):

  • Good thing is that version 6.5.1 does not require a msi installer anymore. You can install from the PowerShell Gallery via PowerShellGet (and the correct version of PowerShell, but we covered that one already):
Install-Module VMware.PowerCLI
    • Use –Scope CurrentUser to use only for this user and no admin permissions required

Install WebCommander:

Download the files from GitHub, for example for the zip file: https://github.com/9whirls/webcommander_walnut/archive/master.zip

Extract the zip and copy to c:\WebCommander. Or use your own location.

The Zip is composed of the following files and directories as subdirs of the master directory:
www/ – These are the files that need to be setup as the web service in IIS. _def is the file that is used to add the locations to the local scripts as defined in sources.json.
powershell/ …the local commands powershells
README.md – Readme file of the project
sources.json – Locations of local and remote scripts when wanting to use the remote script capability.
Note: that is, currently composed of… You never know what the future brings

Note: For scripts depending on your security policy Windows will normally block the files because they were downloaded from an external location, so you will have to unblock these files. Select the file – properties – and press the unblock in the security part at the bottom.

Open IIS Manager to configure the WebCommander site:

  • Remove the default site
  • Add a new site (in this case I used the administrator to connect as to know which user is running, don’t just copy but do what is appropriate for your environment)

Add Webcommandersite

  • select the WebCommander site and open the authentication feature
  • Enable Windows Authentication, and disable Anonymous.

Site Authentication

  • If we now open a browser we will see the initial page

Initial Localhost

When clicking on select a command we can only select the remote commands. use the source.json to define the local locations. For me it was fixed when removing http://localhost/ from the local configuration to read: “local” : “_def.json”,

This one could also help as the _def.json was also a bit empty. Go to c:\WebCommander\powershell\ and execute .\genDefJson.ps1 to recreate the definition json. We should use genDefJson when updating any ps1 scripts.

And voila local also shows up

WebCommander Local also

 

Test drive WebCommander

There are scripts for vSphere actions and Horizon view actions distributed with the Git.

I have seen the following error message pop-up: AuthorizationManager check failed. The following is witnessed, and changed:

  • For some reason the execution policy is back to restricted, Set-ExecutionPolicy RemoteSigned or Unrestricted.
  • with the ExecutionPolicy set to RemoteSigned or Unrestricted, this error may occur if the script or some of the other included scripts is still blocked. From the explorer right-click the file, select Properties and click Unblock. Go through all the files!

Let see if we can get some vSphere information:

  • Add Command vSphere (local)
  • Add the required parameters, go to method to select what you want to do. I just want to see, so listDatastore is my option.
  • And press the play
  • Go to the output if there is a Pass
  • And ….

Pass vSphere (local)

If we want to get rid of the PowerCLI Customer Experience Improvement Program (CEIP) warning in the output. Run the following in Powershell:

Set-PowerCLIConfiguration -ParticipateInCEIP $false

(optionally with -Scope User / AllUsers)

And that’s it for now

– Enjoy WebCommanding throughout the universe!

Sources: labs.vmware.com, virtu-al.net, github.com/9whirls/webcommander_walnut

Introduction to Automation via vCenter Orchestrator – Part 2 Installation and initial configuration

This post is a follow up from my previous post about the introduction to the vCenter Orchestration and it’s architecture within the vCenter system infrastructure. The first part can be found at https://pascalswereld.nl/post/84118414428/introduction-to-automation-via-vcenter-orchestrator.

This part will handle the installation options and the initial configuration before starting to use the system.

Installation

For the installation of vCenter Orchestration we have the following options:

  • Use the vCenter server installation to install vOrchestrator with your vCenter server. This is the default and simplest way.
  • Use the installer on the vCenter installation media to install the vCenter Orchestrator separately from your vCenter server. This is especially for dedicated Orchestration servers or role separation. The installer can be found on the media in the directory vCenter-ServervCO and the application installer is vCenterOrchestrator.exe. This also minimize the hassle of the initial configuration as those are moved to the installed (;-)), just add the vCenter connection in the vSphere web client.
  • Use the vCenter Orchestration Virtual Appliance to deploy as a OVF in your infrastructure.

I have installed vOrchestrator from the vCenter Server installer.  This installs to the vCenter location as defined in the installer input. The directory within the infrastructure structure is called Orchestrator. The installer creates two services, namely the VMware vCenter Orchestrator configuration service for the web configuration, and VMware vCenter Orchestrator Server for the service (where the vCenter and web client connection will be to). These services open web service for port 8283 for the configuration and 8281 for the Orchestrator API. These services are installed in a stopped manual state.

For a small environment with testing or just a small amount of workflows the default vCenter or embedded database can be used. But for a production environment a specific database is highly recommended. I have setup a MSSQL 2008R2 database to be connected to from vCenter Orchestrator. This is an empty database with 1GB space and 10% growth. The databases will be created from the configuration.

Initial Installation

For the initial installation we start up the VMware vCenter Orchestrator configuration service and set this to automatic. After the system starts up, we can connect a browser to https:<servername>:8283/

The initial username and password is, vmware with password vmware. The default password is changed upon connecting. The vmware user cannot be changed.

We see some greens (hooray) and some reds (boooo).

First up check the network settings. These should be okay. Next up import the SSL certificate of the vCenter server. Go to Network – SSL Trust Manager

Fill in the details of your vCenter server at Import from URL. I have installed this on the same host, so https://localhost is sufficient. This needs to be repeated from every vCenter you want to add (a maximum of 20 per Orchestrator).

Next up settings the authentication method. Orchestrator supports LDAP and SSO authentication.  If we want Orchestrator to work from the web client we need SSO setup.

We have to import the SSO certificate in SSL Trust Manager. Import from URL and connect to the SSO on the server. https://localhost:7444 for example.

Next click on authentication and select SSO authentication from the mode drop-down menu.

Add the same URL as the SSL import. For admin and password use the SSO admin credentials. Either Administrator@vsphere.local or any other account you have selected as SSO admin. This account is only used on registering or removing. Click on register.

Next complete the SSO configuration by add a vCO Admin domain and group. Optionally use the filter to filter out the groups.

Click update to complete this step.

Next we are going to set up the database. Go to the database screen.

We select SQL Server, add the SQL username and password. Fill in the name and standard port (1433), database name and instance name. After we save this configuration, we are presented with an option to create the database tables. If you happen to use SSL on the SQL server, you will need to import the database server SSL certificate. We can do this in the SSL Trust Manager by connecting to the server URL and import. For this example I don’t need this step.

Click on the link to create the tables.

Click apply changes.

This changes the red triangle at database to a green circle.

Installing Orchestrator requires a certificate for orchestrator. Depending on how your certificates are managed you either create a signing request and a CA signed certificate or create a self signed.

Click on server certificate. Click create certificate database and self-signed certificate. Enter the relevant certificate details.

For the licenses use the vCenter Server License. Add this information if your not using a licensed vCenter or a pre 4.0 version. Add the host, portnumber 443 Path /sdk and a user that is allowed to connect. Apply changes.

Server availability let’s you set a standalone mode or clustered mode when having multiple Orchestrator servers. We leave it running in standalone mode. We are just introducing to system and are not yet at the moment we need clustering. When using Orchestration in your production environment and administrative processes, you might want to think about leveraging this option.

Start the Orchestration service and set it to automatic. You might want to head over to General – Export Configuration and export this configuration to a back-up location.

Setting up Orchestrator to work with vSphere Web Client.

Login to the vSphere Web client and in home select vCenter Orchestrator.  In vCO home select manage Servers and edit configuration of vCenter server. Fill in the Fixed IP/Hostname of the Orchestrator server in format https://<fqdn server>:8281/vco/api/. Test connection and when all is okay click ok.

Workflows

When the registration is successful you have access to some 394 workflow items, composed of the standard workflows and plugins.

vOrchestrator Client

To perform general administration or edit and create your on workflows you must log in to the client interface. You can start the client from programs-VMware-vCenter Orchestration client (or use the windows key on metro systems to browse your apps).

In the hostname fill in the IP as displayed in the Network tab in vOrchestrator configuration and log in with credentials as defined in the Authentication tab.

This is the most important tool in the set, well next to the Orchestrator interface in the vSphere Web client.

Want to install the client on it’s own, use the standalone installer from the vCenter install media and select client.

 

Next!

Great we have our system ready to use. From this two blog post we know we have it, how it works and how to install and configure the system. Next up is starting to use it. I will follow up with a blog post (maybe not the next, but I will) to actually start using it!

Sources: vmware.com.

Introduction to Automation via vCenter Orchestrator – Yes you can!

Introduction

Automation is something that was started in the industrial revolution. It introduces processes to automate repeating tasks so that quality is maintained or even better (by designing, testing and debugging the process), and minimal or nil human intervention is needed. If you can repeat it, automate it. Well not all, don’t repeat failures, issues and such ;-).
This creates efficient infrastructures, and more important efficient IT departments that can focus on other tasks then repeating that same boring task list all day long.

Automation in the VMware product suites can be done with several products, like vCenter Automation Center for cloud, puppet, scripts, third parties products and the like; but with your vCenter infrastructure there is already a component you can leverage vCenter Orchestrator or vCO for short. With vCO, the VMware administrator can create, using a drag and drop interface, a workflow to for example provision new servers. With steps like deploying a template, customizing the system and installing applications. With plugins additional tasks as creating an Active Directory object, or adding a change request to the IT management system to create the storage needed for you VM’s can be added as well. The prebuild workflows library already has hundreds of out-of-the-box workflows to start using immediately

And the best part, when you installed a licensed vCenter server system, you already have it. Start using it!

Architecture

vCenter Orchestrator can be installed next to vCenter on a Windows system via the vCenter installer (it will by default), or via the vCenter Orchestrator appliance. With the vCenter Server Appliance you don’t have vCO components, you will have to add vCO on a Windows server or add the vCO appliance next to the VCSA. When installing on a Window system use the vCenter server installer to install orchestrator as a component, or do it on a complete separate system. But then prepare your environment by installing a SQL database and starting the vCenterOrchestrator.exe from the vCenter-Server/vCO directory on the install media. Installing the orchestrator appliance is downloading and deploying the OVF. When receiving an IP address the rest of the configuration can be done via a supported web browser.

Orchestrator is composed of three distinct layers: an orchestration platform that provides the common features required for an orchestration tool, a plug-in architecture to integrate control of subsystems, and a library of workflows accessed from the client application (addon to vSphere web client). Other components are the directory services or SSO, the database, and web services for browser access and REST/SOAP API.

image

Orchestrator is an open platform that can be extended with new plug-ins and libraries, and can be integrated into larger architectures through a SOAP or REST API. In my opinion this is the strength of automation tools to leverage the ability to integrate in the infrastructure architecture and not having the orchestration focus on the primary product architecture. With the plugins architecture automation via vCO is reaching beyond the VMware infrastructure, with plugins for AWS, SQL, Active Directory or leveraging SOAP/REST and such.

Release / License

With the release of vSphere 5.5, vCO is at 5.5 as well (well 5.5.1 for the appliance). You can download the integrated installer or the appliance from your MyVMware vSphere 5.5 entitlements.

Orchestration is licensed from the vCenter license. vCO will run either in Player mode: allowing you to run but not edit workflows. Or in Server mode; allows you to run and edit workflows.
Player mode comes with the vCenter Essential and vCenter Foundation licenses, and server mode is leveraged with the standard and higher licenses.

Plugins

The plug-in architecture, you can automate tasks across both VMware and third-party applications.

Plugins can be developed specifically for your needs, by partners. You can find already created plug ins at VMware Solution Exchange. Just log in and search for vCO.

image

 

Going further?

Yes, definitely. But I will follow up in more blog posts.

Sources: vmware.com. solutionexchange.vmware.com.

Learning Puppet – getting the Puppet learning VM

While trying to learn Puppet you should probably first want to know what Puppet is.

What is Puppet and what can we do?

Puppet is an automation configuration management tool from Puppet Labs. It’s supports automation cross-platform, for example in several OS like Windows, Linux but there is also VMware vCenter integration. With its integration with vCenter, Puppet enables provisioning VMware VMs and automatically install, configure, and deploy applications like web servers and databases. For vCloud Director there is also a plugin to furthermore automate deployment of multiple tenants.

For this VMware integration you need the Enterprise edition of Puppet. You can try out a free edition to start and manage 10 nodes for free.

From 11 nodes you will have to buy a additional enterprise license (including standard support) from $99 per node. To more nodes the lower the pricing. See details at http://puppetlabs.com/puppet/how-to-buy.

And… even better at http://docs.puppetlabs.com/learning/ you can get a learning course and a learning VM (vmx or OVF) dowload so you have your Puppet learning lab ready. There is a serverless module and an agent master module to start up your Puppet skills. The learning VM is based on the Enterprise Free edition.

The VM

You can download the learning Puppet VM at http://info.puppetlabs.com/download-learning-puppet-VM.html. Here you will have to register and choose you flavour of download; either the OVF or the VMX (recommended for VMware fusion or workstation). You can install the VM in different flavours of virtualization software like VMware Workstation or VirtualBox.

You can use the console (root and password puppet) to access the system, you can also use SSH. The IP is shown on bootup. The is also a web interface to connect; https://<ip>/. The web interface is for managing your environment and connections. There you will have to log in as puppet@example.com, with the password learningpuppet.

I am using the vmx download and import the vmx in VMware workstation 10 inventory. It is a 1,5 GB download. Extract and put it in a VM folder. Open workstation and from file – open browse to the VMX file. Alternatively you can use explorer and double click vmx if you have a file extension to opening application set-up to workstation.

If your happy with the configuration start up the guest. After boot at the login just wait a few seconds and the IP information with default user passwords are shown.

image

image

You system is now set to further learn Puppet.

– Enjoy learning automation with Puppet!

PowerCLI collection – Changing PSP

On storage controllers in combination vSphere 5.x multi path IO, path selecting policy should be Round Robin (always check if your storage supports this). When receiving the LUN it happens that those PSP are defaulted to Fixed. You can change the PSP policies of all attached storage with this PowerCLI oneliner (Round Robin in the example):

get-cluster -Name “Cluster” | Get-VMHost | Get-ScsiLun -LunType disk | Where-Object {$_.MultipathPolicy -ne “RoundRobin”} | Set-ScsiLun -MultipathPolicy “RoundRobin”

A little caution when you have storage from mixed storage providers (mixed storage controllers without a SVC,Vplex or such layer).

What does this one do:

– get-cluster -Name ”Cluster”; Retreives the cluster information for Cluster. This command can be replaced with get-datacenter. And pipe’s this information to:

– Get-VMHost; retreives the hosts information (one by one in from the cluster). The output is piped to:

– Get-ScsiLun – Luntype disk | Where-Object {$_.MultipathPolicy -ne “RoundRobin}

Get de Scsi Lun of type disk en checks if this object current policy is not equal with Rond Robin. The output is piped to:

– Set-ScsiLun -MultipathPolicy “RoundRobin. The Lun found with previous part is changed to a RoundRobin policy.

The policies can be set to:

RoundRobin, MostRecentlyUsed, Fixed or Unknown.

PowerCLI collection – Adding hosts to inventory

With several projects at several sites I have come along a nice list of powerCLI oneliners or scripts. I wanted a place to have these scripts in a handy catalogue online, maybe share and learn on some blogging skills (or be a little louder than only 140 characters). I know there are probably already place online, but..how can I learn the bloggin. So here goes…

This example is a script that adds 24 hosts to a vCenter inventory. ESXi is installed, vCenter and PowerCLI are setup. PoweCLI has a connection setup to the vCenter (Connect-VIServer -Server <vc> -Protocol HTTPS). No other automation or scripting is in place.

To add 24 hosts to a inventory:

1..24 | Foreach-Object { Add-VMHost hostesxi$_.example.nl -Location (Get-Datacenter DataCenter) -User root -Password “<replace>” -RunAsync -force:$true}

So what does this one do:

  •  1..24; Amount of hosts. This is piped to the command and used by the $_ variabel. Foreach-Object let’s us loop within the 1..24 range so the hosts number will change every loop.
  • Add-VMHost is actually the PowerCli cmdlet to add the host.
  • hostesxi$_.example.nl. Creates hostnames of hostesxi1_.example.nl till hostesxi24.example.nl in the datacenter DateCenter
  • -User and Password are the user and password for access to the ESXi hosts.
  • -RunAsync option is to let the tasks run parallel.