NSX | All about control plane

When I was learning about NSX Control plane, I couldn't find all information in a single pane or page. Information was there but scattered. I thought to gather all the info and put it in below way. I found it better to learn in future. Please don't shy to leave your feedback if you find it useful too.

Below are the components of NSX control plan

• NSX Controller Cluster
• Control Plane Agent (netcpa)
• NSX Logical Router ControlVM

Now Let's explore each of above in detail. 

NSX Controller Cluster

• NSX controllers provide control plane functionality.
• Controllers distribute logical routing network information to ESXi hosts.
• It is responsible for updating ESXi host on the state of the logical network components.
• NSX cluster uses "Sharding" process to distribute workload across NSX controller cluster nodes. Sharding is the action of dividing the NSX controller workload into different shards so that each NSX controller instance has an equal portion of work
• Supported and recommended number of controllers are at least 1 for dev and 3 for production
• A distinct controller node acts as a Master node for given entities such as logical switching, routing and other services.
• When one controller fails, master nodes redistribute the shards to the remaining available clusters.
• The election of the master for each role requires a majority vote of all active and inactive nodes in the cluster. This is the primary reason why a controller cluster must always be deployed with an odd number of nodes.
• It communicate with Control Plane Agent (Netcpa)
• It reduce ARP flooding through ARP suppression.
• NSX controllers creates and save ARP, MAC, VTEP and dynamic route distribution tables.
• NSX controllers are required to implement vxlan networking.
• It is required if you are deploying distributed routers or VXLAN in unicast or hybrid mode.
• If one controller is down, there will be no impact on NSX control plane as other two will service in the NSX cluster.
• If two controllers are down, there will be no impact on running routing configuration but third controller will be read-only and will not support any configuration changes.
• NSX controllers work with CDO (Controller Disconnected Operation) mode, in case all the three controllers are down or any host lost its connectivity with control plan
• NSX Controller communication channel
• Management plane communicates with controller clusters over TCP 443 port number
• The management plane directly communicates with vsfwd in esxi host over TCP 5671 by using RabbitMQ to push down firewall configuration changes
• Control plane talk to the netcpa agent over TCP 1234 to propagate L2\L3 changes
• netcpa propagete these changes to respective routing and VXLAN kernel modules in the ESXi host.

Control Plane User World Agent

• Control plane agent (netcpa) is a TCP client that communicates with the controller using control plane protocol
• It uses SSL to secure the communication with NSX controller instances
• it mediates between controller instance and hypervisor kernel module
• It sends information about network connectivity, IP and MAC address to NSX controllers
• It retrieve configuration information from nsx manager through vsfwd
• Beginning with version 6.3, an auto recovery mechanism has been added to user world agents (netcpa and vsfwd)
• The automatic user world agent monitoring process detect the user world agent in wrong state and try to recover it automatically
• If the user world agent reports a temporary failure due to delayed response to health check then, a warning message is reported in the VMkernel logs
• User world agents are deployed by NSX manager on ESXi hosts through EAM during host preparations.
• Each ESXi hosts in NSX cluster run two user world agents (UWA) that are netcpa and vsfwd.
• Below is the classic screenshot to explain it. Taken from VMware e-book

 

• netcpa auto restart operations:
  

1. netcpa update the global lock periodically
2. An external script checks the heartbeat counter to see whether netcpa failed
3. If netcpa failed, the script restart the daemon and generate the core dump and alerts. Watchdog events are recorded in /var/log/syslog.log

• Verify netcpa agent service in esxi host

Login ESXi host and run the command

 

# /etc/init.d/netcpad status 

Control VM

• It is a component of control plane and not a data path. Mind it:)
• It establish OSPF\BGP neighbour peering.
• It pushes the routing updates to controller cluster
• Control VM is must have component for High availability configuration and with dynamic routing configuration
• NSX CVM communicates with NSX manager and controller cluster
• It sends the routing information to controller clusters
• NSX manager sends LIF information to controller cluster and control VM
• In case, you have static routing then no edge\DLR in HA then you don't need it 
• The DLR control VM can be configured to redistribute IP prefixes for all the connected logical networks into OSPF
• NSX edge pushes the prefixes to reach IP networks in the external networks to the control VM

That's all folks for now but I will keep on adding in the list.

Feel free to ask to cover any specific topic on this.

Thank you,

Team vCloudNotes

Read More

NSX | IP Discovery

Of course, NSX need to know the IP address of any VM and to find the IP address of any VM it uses below methods

- VMware Tool installed on every VM
- DHCP Snooping (Enabled on host cluster)
- ARP Snooping (Enabled on host cluster)

Multiple methods can be used to discover the IP and can be used in below operations\task by NSX Manger

- Firewall Rules
- Spoofguard


IP Discovery with VMware Tools-

VMware tools use thin agent that must be installed on each and every VMs which needs to be protected.
Virtual Machine with installed VMware tool is automatically secured whenever they are started up on any ESXi host having NSX VIB installed.
Protected virtual machines retains the security protection through shutdown and restart and even afrter vMotion move to another host with installed NSX VIBs.
If Vmware tool is not installed then other methods can be used like DHCP and ARP snoopiong.

IP Discovery with DHCP Snooping-

As you already know that DHCP snooping can discover IP without Vmware Tools installed. The four broadcast (DORA) frames that DHCP uses to provide IP address is visible to logical switch or distributed port group as these frames are processed. The assigned IP address then mapped to vNIC. This mapping can be used by NSX Manager to assign firewall rules for this object.

The option to enable and disable the dhcp snooping is available on each cluster.

IP Discovery with ARP Snooping-

It can also be used when there is no VMtools installed in the GuestOS. The ARP request and ARP reply passes throught the logical switch are read and IP addresses associated with vNIC. This association can be used by NSX Manager to apply firewall rules for this object.

Please note that SppofGuard is a feature that in some cases, can prevent ARP snooping in virtual environment.


Happy Learning Friends!




Thank you,
Team vCloudNotes

Read More

PS | Automation can be dangerous!

Hi Folks,

Everyone loves automation. It is very exciting to see that operations is happening automatically. But it will not take much to convert from excitement to graveyard regret if not executed in correctly or in a perfect manner.

I would like to share one example where we were in process of upgrading all NSX edges. As per plan, we selected around 100 edges to upgrade first and then we had to wait for further approval and all.

So we picked the powershell command and executed for those 100 edges. Command is very simple and one liner command. Nothing complex in that but.....

A blank enter key created mess. Let's see how!

Check the command first

In that command, there is source file which is saved with .txt extension. In that file, edge ids needs to be mentioned. isn't it? Then execute the script.

Very simple :)

but what ruin the task is, in that .txt file, edge were mentioned like

edge1
edge2
edge3
.
.
.
and so on and nothing wrong.
But in last row, there was a blank enter after last edge ID.

Script was taking this as an object "ALL" and executed the upgrade action on all other edges as well which were not in that .txt file.LOL!

What a mistake and what a mess!!

Solution is either don't give a blank enter or use .csv file rather than .txt file whenever giving source to an PS script.

Be careful guys. It is operations....LOL.




Thank you,
Team vCloudNotes

Read More

NSX | Plan upgrade with care

Good Morning Folks,

Purpose of this post is to make all of you aware about one of the supported feature of NSX which is no more supported started from version 6.4.4.

Why specifically I am sharing it because it can be a good example on "How you should plan the upgrade".

Feature is "Starting from 6.4.4, 3DES as an encryption algorithm in NSX Edge IPsec VPN service is no longer supported."

Now question is what does it mean and how it will impact production.

What does it mean?
Hope you know that in IPSec VPN tunnel there are two endpoints, one is local and other is remote. 3DES is encryption algorithm which we use to secure the connection between these two endpoints. In place of 3DES, we have AES, AES256 and AES-GCM. We have to select anyone of above because 3DES is no more supported or listed in nsx edge version 6.4.4.

Why it is depreciated?

Because it is not that strong. To elaborate, 3DES designed to auto-negotiate the encryption value to establish the connection, which is not a secure way to make a connection with remote site. Whereas other cipher must be common on both end to establish the connection.

How it will impact the production?

Let's say you have 3DES configured on remote end in phase2 configuration and your local end is configured with AES256. With prior version 6.4.4, IPSec VPN tunnel will continue to work but as you will upgrade NSX edge to 6.4.4, your tunnel will down because both values are not matching any more. You must change the value at remote end to match the value at local site(end).

So please be aware and always check the Documentation of every new version of all products before upgrade.


Feel free to share any thought\doubt\feedback.



Thank you,
Team vCloudNotes

Read More

NSX | What is 3 and 5 tuple value?

This question was asked in one of my interview and I was not that knowledgeable to answer it at that time. I don't want anyone else (who learn NSX and my blog. I have no way to share it with all the world:)) to be unanswered on this question. With this thought, I am writing the answer below. 

It is very small thing but matter a lot while asked in an interview.

It refers to a set of three and five different values that comprise a TCP\IP connection. It include as shown below.

3-Tuple:  The tuple (source IP address, destination IP address, ICMP
      Identifier).  A 3-tuple uniquely identifies an ICMP Query session.
      When an ICMP Query session flows through a NAT64, each session has
      two different 3-tuples: one with IPv4 addresses and one with IPv6
      addresses.

5-Tuple:  The tuple (source IP address, source port, destination IP
      address, destination port, transport protocol).  A 5-tuple
      uniquely identifies a UDP/TCP session.  When a UDP/TCP session
      flows through a NAT64, each session has two different 5-tuples:
      one with IPv4 addresses and one with IPv6 addresses.

I hope who didn't know about this, it will add value in them. 

Be Interview ready and Always! You never know when you will get golden opportunity and that call,you were waiting for since long.

Thank you,

Team vCloudNotes 

Read More

PS | How to read content of any file inside GuestOS without logging in?

I am doing lot around powershell these day. Let's see one more Challenge given and provided solution.

Basically, this challenge belongs to my last blog. Here I was asked to read the content of a file in a VM without accessing RDP. Sound interesting? isn't it?

#Start here
Connect-VIServer vCenter1
$VM = read-host "Enter VM Name " #here is the target vm name

$Chpass = @"
#below command will read and give output for entire file
(Get-Content -Path C:\DRTask\vm-startup-regIpDns_v9.ps1
#below command will read line number 8 and will give you output
    (Get-Content -Path C:\DRTask\vm-startup-regIpDns_v9.ps1 -TotalCount 8)[-1]

"@

Invoke-VMScript -VM $VM -ScriptText $Chpass #-GuestUser "$user" -GuestPassword "$pass"  -ScriptType Powershell

#Start here

Below is output- I got the text in line number8 that that has the IP address in file inside the guest OS.

Basically it read the file with the help of VMware tools in vcenter server.

Cheers! let me know if it worked for you as well. In case of any error please don't shy to put a comment or mail me.




Thank you,
Team vCloudNotes

Read More

Powershell | Transfer file into VM

Hi Guys,

I hope that this post will help many because everyone once in their career might encounter this issue.

The issue is, "RDP for this VM is not working\allowed, how can I transfer this file into this VM". I have seen many guys facing this issue, So, below is the solution

#start here

clear
$VC = Read-Host "Enter the IP address\fqdn of vCenter server"
Connect-VIServer $VC

Write-Host "Enter the requested info please" -ForegroundColor Cyan
Function Collectdata{
Write-Host "Enter the path of source file. For example, C:\temp\transferfile.txt"
$source = Read-Host "Enter the path here"
Write-Host "Enter the destination folder in VM where you want to copy above file. For example, C:\temp"
$dest = Read-host "enter the destination folder path here"
$VM = Read-Host "Enter the VM Name"
$user = Read-Host "Enter the username"
$pass = Read-Host "enter password" -AsSecureString
Write-Host "Thanks to provide all the required info. Tell me the desired action" -ForegroundColor Green
DRSCTransfer
}
Collectdata
Function DRSCTransfer {
echo "Press 1 to transfer the file"
$choice = read-host "Enter your choice here "
if ($choice -eq 1){transfer}

}

Function transfer {
Get-Item "$source " | Copy-VMGuestFile -force -Destination "$dest" -VM $VM -LocalToGuest -GuestUser $user -GuestPassword $pass

}

#end here

Do try this and let me know if any issue.

Thank you,
Team vCloudNotes

Read More

Powershell | How to copy a file from local Machine to GuestOS of a VM

It is very simple. Below is the code-



Get-Item "C:\UcliEvt.log" | Copy-VMGuestFile -Destination "c:\tmp" -VM TestVM -LocalToGuest -GuestUser username -GuestPassword password








Thank you,
Team vCloudNotes

Read More

Powershell | Modify password for user account inside GuestOS of a VM

Today I got this challenge and I did it in below way-

#Start here

$VC = Read-host "Enter your vCenter server name\IP "

Connect-VIServer $VC

$vmName = Read-host "Enter the target VM Name in vCenter "

$UN = read-host "Enter the target username "

$pswd = 'Password' #Enter password here which you want to set in '' mark

$Chpass = @"

`$securePswd = ConvertTo-SecureString -AsPlainText -String $pswd -Force

Get-LocalUser -Name $newUser | Set-LocalUser  -Password `$securePswd -Confirm:`$false

"@

Invoke-VMScript -VM TestVM -ScriptText $Chpass -GuestUser "$UN" -GuestPassword "Asdf@1234" -ScriptType Powershell | Select -ExpandProperty scriptoutput

#End here
Please note that -

"Asdf@1234" is existing password of the guestOS
In case any error, do let me know, will surely help you out




Thank you,
Team vCloudNotes

Read More

vCloud Director is no more… Long Live vCD!

Exciting to witness new chapter in VMware Cloud Journey. Click below for full story-

Cloud Director Services (CDS)









Thank you,
Team vCloudNotes

Read More

NSX | Bit about Firewall

Types of Firewall rules based on protocols and security layer-

General Rules - These rules are applied to the L3, L4 and L7 protocols and fields such as IP addresses, TCP\UDP port numbers and APP-IDs. In addition, vCenter attributes like datacenters and resource pools can be part of the group.

Ethernet Rules - These rules can define a set of MAC addresses as source or destination and enforce policy on L2 protocols. Ethernet rules are enforced before General rules.

Partner Security Services -These rules can define traffic flows to be redirected to partner solutions for additional network introspection.

Firewall rules are managed in centralized manner. Each traffic session is checked against the top rule in firewall table before moving down the subsequent rule in the table. The first rule in the table that matches the traffic parameter is enforced

Types of Firewall rules based on where and who creates them-

• user-defined rules : rules created by administrators
• Internal rules : Rules that enables control traffic to flow for NSX edge services
• Local Rules : rules specific to NSX Manager instance(In cross vcenter NSX deployment)
• Service Composer : rules created through service composer as a part of defined security policy
• Default distributed firewall rules : rules that deals with traffic that does not match any rule 
• Pre-rules: rules created for the NSX edge firewall through the centralized firwall tab. Pre-rules cannot be modified at the edge level

About Reject and Deny action of firewall

• Reject action sends the following responses:
• RST packets for TCP connection
• ICMP unreachable host
• Deny action silently drop the packet just like some RTOs

Applied To field of a firewall rule

There are total 12 objects\destination on which you can apply the rules, those are as below-

1. Datacenter
2. Cluster
3. DvPortgroup
4. Virtual Machine
5. Resource Pool
6. Security Group
7. vAPP
8. vNIC
9. IP Set
10. IP Address
11. Logical Switch
12. Legacy port group.

DFW Enhancement : L7 based enforcement

L7 Firewall also called context-aware security or next generation firewall. Context aware security is intended specifically for east-west cases. However there are no changes how components interact in the next generation firewalls-

* NSX manager talk to vcenter to fetch inventory with vCenter plugin

* If AD is integrated then NSX communicates with AD with AD plugin

*NSX manager sends rules to esxi host which is received by vsfwd installed in the host and then host send these rules to applicable components.

A few components now have additional responsibility

• The Deep packet inspection module is now used to inspect APP-IDs.
• Message bus agent creates filters, configure rules and integrate the above components to collect context
• VSIP module installed on esxi host creates flows based on rules and redirect traffic to the DPI user world engine
• vDPI daemon help find context

Below snippet will explain a lot in one go. Taken from VMware e-book.

Thank you,

Team vCloudNotes

Read More

[Update] Powershell

Hi Guys,

I am still working on the idea to create a Centralize tool for many vCenter Operational tasks with the help of powershell in a time efficient way. I have divided it into three category.

Get or Search Engine ....................In Progress
Set or operations Engine ...............To be initiated
Deploy and Automation Engine....To be initiated

Below is the updated glimpse of the work-

Please give your ideas to make it more exhaustive. I will share once all three steps will be completed.




Thank you,
Team vCloudNotes

Read More

NSX | How NSX provide more security to datacenters?

It secure the data center by addressing one of the key issues in traditional security solutions, which are as below-

1. NSX has visibility not only on virtual datacenter components like VM, ESXi host, portgroups but also within Guest-OS, application and its service hence secure SDDC not only from external attacks but also from within vulnerabilities. Here we know something known as Micro-Segmentation.

For example: if once VM is infected with some viruses\malware or any other malicious software then NSX block the VM and don't allow it to infect other VMs. Below is reference snippet.

First image(from left) is for traditional datacenter and with traditional security.

Second image is with NSX

2. It don't need any in Guest Antivirus agent. Separate antivirus solution is not required if you have Guest-Introspection enabled with NSX. 

Explanation: I have 10 ESXi host in a cluster and 100 VMs. I'll just prepare my ESXi Host for NSX (Let me know if you want to know about this point) and then enable the Guest-Introspection on entire cluster. My all 100 VMs will be secured with enabled antivirus in Guest-Introspection in NSX.

3. Because NSX secure entire cluster so if a VM is vMotioned by DRS or restarted by HA on different host then also it will be secured.


Thank you,
Team vCloudNotes

Read More

NSX | Service Composer

Below is the excellent article on Service composer in NSX.

Very well explained and to the point detail.

Read it completely and feel free to start any discussion.

https://blogs.vmware.com/consulting/tag/nsx-service-composer




Thank you,
Team vCloudNotes

Read More