Wednesday, April 27, 2011

Mounting a Windows share on a VMware ESX server throws error: “Error connecting to <IP or Name> (Connection Refused)”, “14409: Connection to <IP or Name> failed” and “SMB connection failed”.

I was recently asked by an ex-colleague about having an issue with mounting a Windows server share from within an ESX 3.5 server and my first instinct turned out to be wrong so I thought it might be worth while to write up a blog post with the specific error he encountered and stating the resolution.

Problem

  • You’ve logged into an ESX server either through SSH or the console.
  • You create a directory with MKDIR.
  • You make an attempt to mount a windows share but get the following error message:

login as: stats
stats@servervm03's password:
[stats@servervm03 stats]$ su -
Password:
[root@servervm03 root]# mkdir /mnt/esxshare
[root@servervm03 root]# mount -t smbfs -o username=stats //10.10.16.75/stats /mnt/esxshare
Error connecting to 10.10.16.75 (Connection refused)
14409: Connection to 10.10.16.75 failed
SMB connection failed
[root@servervm03 root]# ping 10.10.16.75
PING 10.10.16.75 (10.10.16.75) 56(84) bytes of data.
64 bytes from 10.10.16.75: icmp_seq=0 ttl=127 time=0.305 ms
64 bytes from 10.10.16.75: icmp_seq=1 ttl=127 time=0.257 ms

--- 10.10.16.75 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1017ms
rtt min/avg/max/mdev = 0.257/0.281/0.305/0.024 ms, pipe 2
[root@servervm03 root]#

image

Solution

Since my ex-colleague was trying to mount a Windows Server 2003 share, the first response I had with my coworker was to ensure that he has disabled the Microsoft network server: Digitally sign communcations (always) policy in the Local Security Policy under Local Policies –> Security options.  This turned out to not be the cause because he had already done it.  After spending a few minutes to think about it over the phone, I asked if he had actually enabled the SMB Client service on the ESX server with the esxcfg-firewall command and as it turns out, he hasn’t.  Here’s what he ended up having to do to get it to work:

login as: stats
stats@servervm03's password:
[stats@servervm03 stats]$ su -
Password:
[root@servervm03 root]# mkdir /mnt/esxshare
[root@servervm03 root]# mount -t smbfs -o username=stats //10.10.16.75/stats /mnt/esxshare
Error connecting to 10.10.16.75 (Connection refused)
14409: Connection to 10.10.16.75 failed
SMB connection failed
[root@servervm03 root]# ping 10.10.16.75
PING 10.10.16.75 (10.10.16.75) 56(84) bytes of data.
64 bytes from 10.10.16.75: icmp_seq=0 ttl=127 time=0.305 ms
64 bytes from 10.10.16.75: icmp_seq=1 ttl=127 time=0.257 ms

--- 10.10.16.75 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1017ms
rtt min/avg/max/mdev = 0.257/0.281/0.305/0.024 ms, pipe 2
[root@servervm03 root]# esxcfg-firewall -e smbClient
[root@servervm03 root]# mount -t smbfs -o username=stats //10.10.16.75/stats /mnt/esxshare
Password:
[root@servervm03 root]#

image

**Notice how the console now asks for the password of the login that is being used to the access the share.

What I always like to do at this point is to test the connection to the share with the touch command as such:

[root@servervm03 root]# touch /mnt/esxshare/testfile.txt

[root@servervm03 root]#

image

Once the command successfully executes, open up the share and confirm that the file was indeed created:

image

If you see the file as shown in the screenshot above, you’ve successfully created the mount.

Tuesday, April 26, 2011

Backup and Restoring Active Directory Partitions in Windows Server 2003

As mentioned in one of my earlier posts, I’ve been performing some tests with Active Directory restores to provide myself with a refresh of the procedure as I haven’t done it for awhile and just so I won’t need to do this again in the future, I thought it would be beneficial to myself to document the process so I have screenshots and steps to reference to in the future.

We’re all familiar with the native NTBackup utility that Windows Server 2003 provides and it is important to remember that whenever a change is to be made to your directory service, always remember to perform a backup of your domain controller.  I almost always perform a backup of more than one domain controller whenever I’m about to make any changes.

Backup Process

Start up NTBackup on your domain controller and select the System State from the list of items to be backed up:

image

Click on the Start Backup button and wait for the backup to complete:

image

Now that we have a backup completed, let’s begin mimicking an unintended change in Active Directory.  For the purpose of this example, we’ll be deleting the Public Key Services node in the Configuration container:

image

image

image

image

image

As shown in the screenshot above, the Public Key Services node in the Configuration container has been deleted (bye bye CA services) from Active Directory.

Restore Process

Prior to actually proceeding to restore the System State that was backed up, we will need to restart the domain controller in Directory Services Restore Mode either locally or remote.  Locally is done by simply hitting F8 during start up while for remote, we’ll need to edit the start up options in the boot.ini file:

image

To force the domain controller to restart in Directory Services Restore Mode simply append the following to the end of the boot.ini file:

/SAFEBOOT:DSREPAIR

... which will change your boot.ini file to look something like this:

[boot loader]

timeout=30

default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS

[operating systems]

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003 Enterprise x64 Edition" /noexecute=optout /fastdetect /SAFEBOOT:DSREPAIR

Once the changes have been completed, proceed with rebooting the domain controller:

image

If you have access to the console, you’ll notice that once the domain controller begins to start up, the wallpaper will look different:

image

Wait for the boot up process to complete and you’ll be able to remote desktop to the domain controller:

image

It’s important to note that you will need to login with the Active Directory Restore Mode password and if for some reason you need to have forgotten it, use the following KB article to reset it: http://support.microsoft.com/kb/322672.

image

Once you’ve successfully logged in, you’ll be promopted with the following warning/informational message:

image

One of the first things I do once I successfully get into the domain controller is to remote the /SAFEBOOT:DSREPAIR switch from the boot.ini file:

image

Once you’ve completed the change to the boot.ini file, continue with restoring the the backed up System State to the domain controller:

image

image

image

image

image

image

image

image

image

image

Note that if the domain controller you are restoring is the only domain controller in your environment, you will need to select the last option:

When restoring replicated data sets, mark the restored data as the primary data for all replicas.

image

image

image

image

Once the backup is complete, click on the Close button and remember to select No when asked to reboot:

image

Marking an object or objects as authoritative

Start up NTDSUTIL:

image

image

Then execute the following command:

authoritative restore: restore subtree "CN=Public Key Services,CN=Services,CN=Configuration,DC=contoso,DC=local"

image

You’ll be prompted with a confirmation message asking if you’re sure you would like to perform this authoritative restore so continue by selecting Yes and you will see the following output:

C:\>ntdsutil

ntdsutil: authoritative restore

authoritative restore: restore subtree "CN=Public Key Services,CN=Services,CN=Configuration,DC=contoso,DC=local"

Opening DIT database... Done.

The current time is 04-20-11 14:57.04.

Most recent database update occured at 04-19-11 22:24.00.

Increasing attribute version numbers by 100000.

Counting records that need updating...

Records found: 0000000008

Done.

Found 8 records to update.

Updating records...

Records remaining: 0000000000

Done.

Successfully updated 8 records.

The following text file with a list of authoritatively restored objects has beencreated in the current working directory:

ar_20110420-145704_objects.txt

None of the specified objects have back-links in this domain. No link restore file has been created.

Authoritative Restore completed successfully.

authoritative restore:

image

Once the restore is complete, proceed to reboot the server:

image

Verifying the restored object

Once the domain controller has been rebooted, open ADSIEdit, connect to domain controller you restored the partition on and navigate to the node that contains the restored object:

image

As seen in the screenshot above, while DC1 has the Public Key Services node in the Configuration container restored, the changes haven’t been replicated to DC2 yet:

image

What’s also worth noting is that opening Replication Monitor and forcing a replication shows that certain partitions aren’t replicating properly:

image

What’s important at this point is not to panic because it does take a bit of time for the 2 domain controllers (or more if you had more than 2) to resolve the conflicts they seen between each other after a restore.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

For those who are interested in how this conflict is resolved, domain controllers actually have an attribute named uSNChanged that it uses to track which domain controller has the most up-to-date value for the attribute and just so I don’t accidentally give the wrong explanation for this attribute because of my poor memory, I will simply provide a copy and paste from Microsoft:

When a domain controller modifies an object it sets that object's uSNChanged attribute to a value that is larger than the previous value of the uSNChanged attribute for that object, and larger than the current value of the uSNChanged attribute for all other objects held on that domain controller. As a consequence, an application can find the most recently changed object on a domain controller by finding the object with the largest uSNChanged value. The second most recently changed object on a domain controller will have the second largest uSNChanged value, and so on.

Reference: http://msdn.microsoft.com/en-us/library/ms677627.aspx

This means that if you open up the properties of the object you’re restoring, you can compare the uSNChanged values for the same object on 2 domain controllers to determine which one will eventually take precedence against the other:

image

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

Depending on your environment, it may take some time before the restored object shows up on the other domain controllers so what I typically do is to force a replication periodically in Replication Monitor and use the results of the replication to determine whether the changes have been replicated already.  Just in case the previous sentence wasn’t clear, what I basically mean is:

  • If you see problems in the forced replication, it means you still need to wait.
  • If you see a successful replication when you force a replication, it means the restore has been replicated.

Once no errors are reported in the replication, you should be able to see the restored object within ADSIEdit:

image

Hope this helps anyone out there who have not seen what the backup and restore process looks like.  Reference articles from Microsoft can be found here:

Performing an Authoritative Restore of Active Directory Objectshttp://technet.microsoft.com/en-us/library/cc779573(WS.10).aspx

Appendix A: Forest Recovery Procedureshttp://technet.microsoft.com/en-us/library/cc781218(WS.10).aspx

NTDSUTILhttp://technet.microsoft.com/en-us/library/cc755915(WS.10).aspx

Sunday, April 24, 2011

Attempting to enable a user for Microsoft Lync Server 2010 throws the error: “Enable-CsUser : Cannot move legacy user in enable operation. Use the Move user cmdlet instead.”

Problem

I recently deployed Microsoft Lync Server 2010 in a small environment and was told that Microsoft LCS and OCS was never deployed so I went ahead and treated the environment as a greenfield but as soon as I began to enable users for Lync, I noticed that some users would throw the following error when I try to enable them in PowerShell:

PS C:\Users\tluk-admin> enable-csuser -identity domain.local\dtrott -registrarpool ly
ncpool01.domain.local -sipaddresstype samaccountname -sipdomain domain.com
Enable-CsUser : Cannot move legacy user in enable operation. Use the Move user
cmdlet instead.
At line:1 char:14
+ enable-csuser <<<<  -identity domain.local\dtrott -registrarpool lyncpool01.ccs.in
t -sipaddresstype samaccountname -sipdomain domain.com
    + CategoryInfo          : InvalidOperation: (:) [Enable-CsUser], InvalidOp
   erationException
    + FullyQualifiedErrorId : ProcessRecord,Microsoft.Rtc.Management.AD.Cmdlet
   s.EnableOcsUserCmdlet

PS C:\Users\tluk-admin>

image

As some of you have already guessed, the reason why I am getting this error is because the user I was trying to enable for Lync was at one point enabled for Microsoft OCS and therefore the Enable-CsUser cmdlet is recommending that I try using the cmdlet to move a legacy user instead.  It would have been easy to get these users over if the old OCS environment still existed but that wasn’t the case for this organization.

Solution

What I ended up doing was to open up adsiedit.msc, navigate to the user’s object, open the properties, and removing or setting the following attributes to “<not set>” and “0”:

    1. msRTCSIP-ArchivingEnabled
    2. msRTCSIP-FederationEnabled
    3. msRTCSIP-InternetAccessEnabled
    4. msRTCSIP-OptionFlags
    5. msRTCSIP-PrimaryHomeServer
    6. msRTCSIP-PrimaryUserAddress
    7. msRTCSIP-UserEnabled

Here’s an example of a user who still had legacy OCS attributes populated for their account:

image

Here’s an example of a user account that had the legacy OCS attributes removed:

image

I’d have to say that while this solution works, it properly doesn’t scale well if you had more than 50 users in the organization so if you fall into that situation, I would suggest that you make an attempt to script the procedure for everyone in Active Directory (make sure you run the script before you enable anyone for Lync).

Friday, April 22, 2011

Restoring Active Directory OUs, objects, and partitions throws the error: “This application has failed to start because SAMSRV.dll was not found. Re-installing the application may fix this problem.”

As I prepare to plan for a infrastructure upgrade for a client, I decided to build a small lab environment to do some tests with directory services restoration as it’s been years since I’ve had to do a restore and seeing how most projects never seem to go as anticipated, I figure it would be best to be as prepared as I could through this refresher exercise.  While performing test restores with my two test domain controllers, I noticed an odd error message that I would continuously receive when using the NTDSUtil command to mark OUs, objects or partitions as authoritative when I execute it in the Windows Support Tools directory.  The following is the prompt with the error is what I would receive:

image

ntdsutil.exe – Unable To Locate Component

This application has failed to start because SAMSRV.dll was not found. Re-installing the application may fix this problem.

image

C:\Program Files (x86)\Support Tools>ntdsutilntdsutil: authoritative restore

authoritative restore: restore database

Opening DIT database...

Failed to recover database from external backup. Error 0x7f(The specified procedure could not be found.).

Authoritative Restore failed.

Error 8000ffff parsing input - illegal syntax?

authoritative restore:

image

I originally didn’t know what was causing this because I would try rebooting the server, complete a restore of the system state again, execute ntdsutil with an authoritative restore command and it would work.  Performing some searches on Google returned some results that lead me to the following Symantec KB:

http://www.symantec.com/business/support/index?page=content&id=TECH30923

Though I went ahead to try and delete this key, the restore would continuously fail.

It was not until after going through this process 5 to 6 times that I noticed that it would fail whenever I executed the ntdsutil.exe command in the C:\Program Files (x86)\Support Tools folder.  The following screenshot is when I executed the command in a regular Windows Command Prompt successfully:

image

Note: I understand that the example above shows I restored a the domain partition but take my word that the difference doesn’t matter and that the reason why it worked was because I wasn’t in the support tools folder.

I’m not sure if this problem is specific to the server OS I used but just in case you’re encountering the same issue as I did, the following is the information about the domain controller:

Server OS: Windows Server 2003 R2 64-bit

Service Pack: SP2

Hotfixes and Updates installed:  All patches installed up to April 22, 2011

Forest Functional Level: Windows 2003

Domain Functional Level: Windows 2003

Restoring Active Directory no matter the whole database, OUs, objects or partitions is always nerve racking so I hope this post will be able to help anyone who may encounter this problem during a restore.