virtuallyGhetto

Automation

Quick Tip – Certificates in Apple Keychain causes Terraform init to fail with Registry service unreachable

by 1 Comment

I have been struggling with an interesting Terraform issue on my MacOS system where running the "init" operation would throw the following error:

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...

Registry service unreachable.

This may indicate a network issue, or an issue with the requested Terraform Registry.

Error: registry service is unreachable, check https://status.hashicorp.com/ for status updates

This was extremely frustrating to debug which I had filed a Github issue here. From what I have gathered, this actually had nothing to do with connectivity to the HashiCorp endpoint which works perfectly but probably was related to some other issue. What was even more strange was that using "sudo" which another user reported in an older issue allowed the operation to go through. I was also not having this problem on my other MacOS system, so I knew this was probably environmental but was running out of ideas to try.

I took another look this past weekend while doing some testing and I stumbled onto this thread here which the user found the real root cause. It looks like certain certificates within Apple Keychain Access, possibly related to Microsoft Remote Desktop that have expired was actually causing the problem. When I took at look at the Keychain Access login->certificates, I saw a number of certificates which had expired but were still marked trusted. After removing these entries (although this can be automated using the security utility, it was not trivial given the lack of arguments to quickly list out expired certificates), that I simply used the UI to delete the entries.

Once all the expired certificates were removed, I was able to successfully perform the Terraform init operation! I have already shared this update in my Github issue and hopefully this error message can be improved in the future as it was very miss-leading on the actual issue.

Quick Tip – HTTPs now supported with wget on ESXi 7.0

by 1 Comment

This morning I needed to install the USB Network Native Driver Fling for ESXi and realized that offline bundles can not be installer over a remote URL. I figure I would download it directly to my ESXi host using wget. As soon as I hit the enter key to send the command I quickly remember that this will fail as the version of wget in the ESXi Shell does not support HTTPs. To my surprise, it actually worked! 😍

[[email protected]:~] wget https://download3.vmware.com/software/vmw-tools/USBNND/ESXi700-VMKUSB-NIC-FLING-34491022-component-15873236.zip
Connecting to download3.vmware.com (184.29.106.37:443)
ESXi700-VMKUSB-NIC-F 100% |**************************************************************************************************************************************************************************| 341k 0:00:00 ETA

This looks to be an enhancement in ESXI 7.0 and you no longer will get the "bad address" error when specifying an HTTPs URL as shown in the example below when running against the latest ESXi 6.7 Update 3 release.

[[email protected]:~] wget https://download3.vmware.com/software/vmw-tools/USBNND/ESXi700-VMKUSB-NIC-FLING-34491022-component-15873236.zip
wget: bad address 'download3.vmware.com'

Thank you to the Engineer who fixed this, this was a very pleasant and welcome surprise that I know will also be helpful to folks who automate using ESXi Scripted Installations.

Updating the VSAN HCL & Release Catalog DB using VSAN API

by 2 Comments

Both the VSAN Hardware Compatibility List (HCL) and the VSAN Release Catalog database which provides VSAN build recommendations should be updated periodically to ensure that you have the latest VSAN recommendations from VMware. In addition to using the vSphere UI to perform these update, customers can also automate either of these tasks using the VSAN Management API which can be consumed using any of the supported VSAN Management SDKs including PowerCLI.


I recently had a question about which VSAN API to use to update VSAN Release Catalog.

[Read more...] about Updating the VSAN HCL & Release Catalog DB using VSAN API

Full OVA/OVF property support coming to Terraform provider for vSphere

by 4 Comments

Terraform is one of the most popular Infrastructure as Code (IaC) tool out there today and it should come as no surprise there is Terraform provider for vSphere which many of our customers have been using. In fact, VMware just recently released a couple more new providers (here and here) supporting VMware Cloud on AWS and NSX-T solutions respectively.

Although I have used Terraform and the vSphere provider in the past, it has not been my tool of choice for automation as it still lacks a number of basic vSphere capabilities which I require on a regular basis. The most common one being the ability to deploy a Virtual Appliance (OVA/OVF) which has been my biggest barrier and I know this has been a highly requested feature from the community as well.

In early May of this year, I noticed that v1.18 of the vSphere provider finally added support for OVA/OVF deployment and I was pretty excited to give this a try and may even have been the first to kick the tires on this feature? Although OVA/OVF support was added, it looks like support for customizing OVF properties which is commonly included as part of an OVA/OVF would only possible if you are cloning from an existing imported OVA/OVF image. One of the most common use case is to import an OVF/OVA from either your local computer or from a URL and it looks like this use case was not possible.

I filed two Github issues, one for supporting OVF properties for initial OVA/OVF deployment and another regarding a bug I ran into when importing OVA/OVF from a remote URL. Just yesterday, I got the good news that my feature request has been completed and I was given an early drop of the vSphere provider to try out this feature. I may have also hinted to the Engineering team to use my popular Nested ESXi Appliance OVA as a reference test implementation as I knew this was something many customers will want to deploy 🙂

UPDATE (06/23/20) - Support for OVA/OVF properties is now available as part of 1.20 of the Terraform Provider for vSphere

[Read more...] about Full OVA/OVF property support coming to Terraform provider for vSphere

Extending VMware Cloud on AWS Notifications using the Notification Gateway API

by 2 Comments

The VMware Cloud Notification Gateway (NGW) Service was launched back in May 2019 and is used to communicate important customer-facing notifications which can be delivered across a number of different communication channels as shown in the diagram below.


Of all the different communication channels, I think one of the most interesting one is the ability to send an outgoing webhook based on a specific VMware Cloud Event. In fact, this was the very first thing that caught my attention when I had first learned about the NGW Service from Nancy Cheng, the Product Manager for this service.

You can probably guess why I was so excited for this feature as it mimics a similiar capability to our VMware Event Broker Appliance (VEBA) solution. This not only enables our customers to consume other public cloud services that support webhooks but it also opens up the door for more advanced integrations, more on this at the end of this blog post 😀

As of publishing this blog post, there are over 75+ VMware Cloud Events which customers can subscribe to such including when a new SDDC is created or deleted, a new ESXi host has been added either manually or automatically via our Elastic DRS (eDRS) Service, SDDC maintenance notices to subscription reminders to just name a few. Although the default email and UI channels are great, many customers would also like to receive these notifications using other popular communications channels such as Slack or Microsoft Teams.

To help demonstrate the webhook functionality of the NGW Service API, I have created a PowerShell Module for VMware Cloud Notifications called VMware.VMC.Notification which is also published i then Microsoft Powershell Gallery. The module contains the following functions:

  • Connect-VmcNotification
  • Get-VmcNotificationEvent
  • Get-VmcNotificationWebhook
  • Test-VmcNotificationWebhook
  • Remove-VmcNotificationWebhook

[Read more...] about Extending VMware Cloud on AWS Notifications using the Notification Gateway API

Setup custom login banner when logging into a vSphere with Kubernetes Cluster

by Leave a Comment

While working on my PowerCLI module for enabling workload management for a vSphere with Kubernetes (K8s) Cluster, I came to discover a pretty cool feature that is only available when using the vSphere with K8s API to enable Workload Management on a vSphere Cluster.

As part of the enablement spec, there is a new property called login_banner. Taking a closer look, this property allows you to specify a custom message that would be displayed as part of the initial login to your vSphere with K8s Cluster using the vSphere kubectl plugin. This is similar to an SSH login banner which can be used to provide internal disclaimers and/or additional instructions for your end users.

Here is an example of what the login banner can look like. Yup, vSphere with K8s supports emojis or rather the terminal you are using to login can potentially render emojis 😀


The good news is that I have already added this feature into the new New-WorkloadManagement function and you can specify a message by adding the -LoginBanner parameter.

For those interested in rendering emojis within their banner, you can take a look at the following example and you can find the complete list of emoji unicodes here.