The computer crashes and reboots for no apparent reason, and now users get an error message when trying to access any of the file or printer shares on the server/computer.  This is related to a registry setting, the Security Event Log filling up, and a possible bug in 2008 R2 that doesn’t allow users to access the server even after the Security Event Log issue has been resolved.

Upon further inspection, the Windows Security Event Log is full and HKLM\SYSTEM\CurrentControlSet\Control\Lsa\CrashOnAuditFail = 1.  You resolve this issue by right-clicking the Security event log, selecting properties, then doing one of the following:

1. Increase the “Maximum log size”
2. Allow the Security event log to “Overwrite events as needed (oldest events first)”
3. “Archive the log when full, do not overwrite events”

This has solved the Security Event Log issue, but now users report they can no longer access \\computername, or any of the shares or shared printers on that machine.   You (a member of the administrators group on the machine), and other administrators on the machine can access shares and shared printers without a problem.

Regular users receive this error:

\\computername is not accessible.  You might not have permission to use this network resource.  Contact the administrator oft his server to find out if you have access permissions.

Logon failure: user account restriction.  Possible reasons are blank passwords not allowed, logon hour restrictions, or a policy restriction has been enforced.

Cause:

The Windows Security Event Log has filled up, causing the server to crash.  This was caused by the following registry value:

HKLM\SYSTEM\CurrentControlSet\Control\Lsa\CrashOnAuditFail = 1

This could have been set by a security template, group policy, or manual registry change.  In any case, the server crashed because it could no longer log audit-able events to the security log.  Users can not access this server until this issue is resovled (see steps 1-3 above).  But even after the Security log is happy again, users may still receive the “\\computername is not accessible” error.

Resolution:

1. Increase the “Maximum log size” to 1GB or more
2. Allow the Security event log to “Overwrite events as needed (oldest events first)”
3. Set HKLM\System\CurrentControlSet\Control\Lsa\LimitBlankPasswordUse = 0
4. Reboot
5. Set HKLM\System\CurrentControlSet\Control\Lsa\LimitBlankPasswordUse = 1
6. Reboot

This took hours to figure out.  Hopefully this post saves someone else some time.

Related Event Log IDs

<Event ID 4625>
Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          4/30/2010 4:20:59 PM
Event ID:      4625
Task Category: Logon
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      <computer where this event was logged>
Description:
An account failed to log on.

Subject:
Security ID:         NULL SID
Account Name:        -
Account Domain:      -
Logon ID:       0×0

Logon Type:                3

Account For Which Logon Failed:
Security ID:         NULL SID
Account Name:        <username>
Account Domain:      <domain>

Failure Information:
Failure Reason:      Unknown user name or bad password.
Status:              0xc000006e
Sub Status:          0×0

Process Information:
Caller Process ID:   0×0
Caller Process Name: -

Network Information:
Workstation Name:    -
Source Network Address:    <IP address of client>
Source Port:         3089

Detailed Authentication Information:
Logon Process:       Kerberos
Authentication Package:    Kerberos
Transited Services:  -
Package Name (NTLM only):  -
Key Length:          0

This event is generated when a logon request fails. It is generated on the computer where access was attempted.

The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).

The Process Information fields indicate which account and process on the system requested the logon.

The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
</Event ID 4625>

<Event ID 4776>
Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          4/30/2010 4:20:55 PM
Event ID:      4776
Task Category: Credential Validation
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      <computer where this event was logged>
Description:
The computer attempted to validate the credentials for an account.

Authentication Package:    MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Logon Account:  <username>
Source Workstation:  <client computername>
Error Code:     0xc0000064
</Event ID 4776>

13%: Amount of printed pages wasted by banner pages: 13% (1 out of 8 pages is usually thrown away recycled).

1,050,000 pages: Average yearly print volume in my dept (yes, over a million)

$932: Money wasted on paper to print banner pages each year.  That’s just the paper cost, and does not include toner, printer maintenance, etc.

Call them what you want: banner pages, separator pages, job sheet, etc.  These things are a waste of money.  I’m looking for solutions to eliminate banner pages from my department.  So far, I’ve found two options.

The on-printer option

The Lexmark TS6545dn is relatively cheap ($1500), fast (55ppm), and has a function called “Reserve Print” that allows printouts to be held on the printers hard drive until they are printed via the printer’s LCD interface.

The HP P4515n is also relatively cheap (~$1500), fast (62ppm), and has “Job Storage Features”, which include “Personal Jobs”.  This feature is a bit nicer in that it automatically uses the username (you don’t have to set it in the driver config like the Lexmark), and automatically deletes the job after it prints.  The HP is a bit worse in that there are 3-4 more steps necessary to retrieve printouts.  A pin (4-digit code) is still necessary, but can be set to a default like “0000″, which is acceptable for my uses.  HP’s Web Jetadmin may be able to work around the quirks (allowing multiple copies at the printer).  If this is true, HP will be the clear winner.  I’ll updated this post as more details become available.

Pros

1. Free!  The feature is built in to the printer and requires minimal additional hardware (an internal hard drive is recommended for print job storage).
2. Easy to use!  Walk up to the printer, find your username, find the job you want to print and press go.

Cons

1. At the printer, you can print multiple copies of your document, even if you just sent 1 copy to the printer.  I support student printing and need a way to account for who prints what and how much.  This detraction is present on both Lexmark and HP printers.
2. Lexmark only: Print job is not automatically deleted after you print it.  This may not be a “con” in your case, but it is in mine.

The add-on option

HP, CZ, and others have release station software.  This runs on a computer near the printer and is used to release print jobs from the queue on an as-needed basis.  The best implementation I’ve seen uses an all-in-one touch screen computer hung on the wall above the printer.  Of course that can get expensive.

Pros

1. Use it with any printer since it isn’t printer specific.
2. Easy to use.  Touch the print job you want and press release.
3. Use to load-balance printers.  Choose which printer you want to release your job to (ex: if there are 3 printers in the same room)

Cons

1. Expensive.  The touch screen all-in-one PC, plus related software licenses quickly tops $1000.

Do you have/use an option I don’t know about?  Please share your knowledge by leaving a comment below.!

Environment

200 PCs (staff and lab machines) running Windows XP Pro, all with at least 40GB free.

Goal

Utilize more of the storage we have already paid for.  Most workstations are online 24×7, and have unused disk space (20GB to 200GB).  I would like to pool the storage in a highly redundant fashion (ex: 4 machines contain the same data so 1 can be powered off with no impact on data accessibility) to present a large block of usable space.

Possible technologies to use

· DFS

· iSCSI+ZFS

· iSCSI+software RAID

Possible uses of combined, redundant, pooled storage

· CIFS shares

· VTL storage

· large temporary storage for student use

· large DB backend (probably not a great use for this, but given the number of spindles, it has potential)

Benefits

· highly redundant

· LOTS of spindles

Possibly helpful links
http://www.faqs.org/patents/app/20080320097
http://www.datacore.com/
http://msdn.microsoft.com/en-us/library/bb524801(VS.85).aspx

http://www.gluster.org

iSCSI Target software for Windows
http://www.iscsicake.com/
http://www.emboot.com/products_sanFly.htm
http://www.kernsafe.com/Default.aspx

Installing Windows 2003 Server’s Network Monitor on Windows XP

1. Install Network Monitor on a Windows 2003 server if it is not already installed.
   1. Start -> Control Panel -> Add/Remove Programs -> Add/Remove Windows Components -> Management and Monitoring Tools -> Details -> [check] Network Monitor Tools -> OK -> Next -> Finish
2. On the XP workstation, install the network capture driver
   1. Start -> Run -> CMD -> OK -> netcap -> the capture driver is now installed, hit the space bar to stop the packet capture
3. Copy the netmon folder from the System32 folder on the server (%windir%\system32\netmon) to the System32 folder on the workstation (%windir%\system32\netmon).
   1. Be sure to grant system and administrator change permissions on the netmon folder and all the files/folders it contains, otherwise netmon.exe won’t start. 
   2. Be sure the “parsers” folder is included when you copy the netmon folder. Drag-and-drop copy keeps this, as does xcopy, but regular “copy” from the command prompt does not.
4. Copy nmsupp.dll from the server (%windir%\system32\nmsupp.dll) to the netmon folder on the workstation.
5. Run netmon.exe

1. On VMware virtual machine, set Exchange and SMTP services to manual
2. Shut down virtual machine
3. Create a backup of the VMDK and related files for OS (C:\) partition
4. Add and IDE disk to the virtual machine to enable IDE drivers in Windows and make the Hyper-V converted disk easier to boot.
5. Remove Exchange’s mail store VMDK disk (E:\) from the virtual machine’s configuration to ensure no data is changed on that partition.
6. Disable networking in VMware console so that any changes made are confined to the local machine.
7. Boot working copy of Exchange OS partition
8. Log on as local admin
9. Uninstall VMware tools and reboot VM when prompted
10. Cancel “Found new hardware wizard” and do not reboot when asked
11. Verify IDE drive is present & drivers loaded
12. Append C:\windows\system32\hal.dll and ntoskrnl.exe with .bak extensions and copy versions from C:\windows\servicepackfiles\i386 to replace the versions in the system32 folder.
13. Run prepvm.vbs created by Chris Wolf
14. Shut down the server
15. Copy modified VMDK files for OS partition (one 20GB and one 1KB) to LUN on IBM SAN for Hyper-V server (arch-host-09) to use
16. Convert VMDK file to VHD file using the VMDK to VHD converter from vmtoolkit.com
17. Link VHD to VM in Hyper-V and boot the file. The following steps were used for troubleshooting.
    1. Booted to Windows 2003 server OS CD to run recovery console
    2. Added a line in boot.ini with arguments “/bootlog /sos /safeboot:minimal” which enabled the system to boot into safe mode.
    3. Rebooted server and chose normal boot-up
    4. Copied halaacpi.dll from C:\windows\servicepackfiles\i386 to overwrite C:\windows\system32\hal.dll
    5. Reboot server normally
    6. Opened “System” in Control Panel to remove reference to the old hal.dll in the Computer properties.
    7. Rebooted system
18. Installed Hyper-V add-ons and rebooted.

Surprisingly, that’s all it took.  Step 17.4 took a little while to figure out because I kept getting an error message about ACPI-compatible hal.dll when trying to install the Hyper-V add-ons.

1. Teamviewer – absolutely my #1 pick for supporting friends & family. Free for personal use, very fast, easily bypasses most firewalls, routers, NATs, etc. 
   My implementation for home use: http://www.aaronpalermo.com/help
2. VNC is a free remote control software package. Many versions exist (UltraVNC, TightVNC, RealVNC, etc.) The main drawback here is TCP port 5900 has to be open and routed to the requestor’s PC. This can be a real hassle if the requestor is using NAT, and/or a cable modem router/firewall, and/or a 3^rd party software firewall. The easiest way I have found to get around this is using UltraVNC’s SingleClick add-on. It is a lot easier to ensure you can listen on port 5500 than trying to talk someone else through configuring their router(s) and firewall(s), especially if they are the type of person who has to call you for help in the first place.
   My implementation for work: http://staff.cs.tamu.edu/palermo/help
3. Remote Assistance built into XP and Vista works well if you have it preconfigured. TCP port 3389 has to be listening and routed to the requestor’s PC, and you have to have a remote assistance invitation, or be a admin on the same domain as the requestor. Also, you have to know the requestor’s IP address (yes, a quick visit to www.whatismyip.com fixes this, but just one more thing to talk a person through)
4. MSN messenger, Netmeeting (called Windows Meeting Space in Vista), Skype, and other instant messenger programs have remote assistance (also called desktop sharing) functions built in, but if you don’t already use one of those, then do you really want to create yet another login & password to remember?
5. Other commercial options like webex and GoToMeeting exist for remote support, but they range from slightly pricy to price gouging. These may be excellent options for enterprise level support, but not for personal or small business use.

Dim array1(1,1)
array1(1,0) = split("hello world"," ",-1,1)

Using the code above, “hello” is actually stored in array(1,0)(0), not array1(1,0).  array1(1,0) actually contains a 2 element array!  The way to get around this is use a temp array to store the output of the split function, then a for loop to copy data from the temp array into array1.
If you have a pre-defined set of data you want to collect (ex: computername,userid,timestamp,eventtype), then an array of objects may be easier to use, and easier to understand if someone else wants to read your code.  A small example of an array of objects that took the place of  a 2D array for a recent project:

Class AuditEvent
	Public ComputerName
	Public Username
	Public Timestamp
	Public EventType
End Class
Dim objAuditEvent()
ReDim objAuditEvent(20)
for i=0 to ubound(objAuditEvent)
	Set objAuditEvent(i) = New AuditEvent
next
objAuditEvent(1).ComputerName = "L210C99"
objAuditEvent(1).Username = "user"
objAuditEvent(1).Timestamp = "02/11/2008 12:00:17"
objAuditEvent(1).EventType = "Logon"
ReDim Preserve objAuditEvent(1)
wscript.echo objAuditEvent(1).ComputerName
'Clean up objects
for i=0 to ubound(objAuditEvent)
	Set objAuditEvent(i) = Nothing
next

The easy way 

The easiest is with makecert.exe, but that doens’t provide you with a trusted cert.  Our departmental CA runs on Solaris and uses OpenSSL.
Chris Blankenship’s post “How to Pre-Create an EFS Certificate”
Downloaded makecert.exe from here
Below are the makecert options that I use to create certificates compatible with EFS and File Recover.  Many thanks to Chris Blankenship’s post (see link above).

For the users:
makecert.exe -n “CN=User Name,OU=Computer Science,O=Texas A&M University,L=College Station,S=TX,C=US” -pe -sky exchange -m 96 -a sha1 -eku 1.3.6.1.4.1.311.10.3.4 -len 1024 -m 1200 User_EFS.cer

For the admins:
makecert.exe -n “CN=Admin Name,OU=Computer Science,O=Texas A&M University,L=College Station,S=TX,C=US” -pe -sky exchange -m 96 -a sha1 -eku 1.3.6.1.4.1.311.10.3.4.1,1.3.6.1.4.1.311.10.3.4 -len 1024 -m 1200 EFS_and_Recovery.cer

The way I do it

I already have a trused Certificate Authority (OpenSSL on UNIX), so my original goal was to create a EFS and File Recovery certificate using the existing CA.  This method can also be used to generate certificates in Windows using OpenSSL.  OpenSSL binaries for Windows can be found here.
The following 3 lines are the OpenSSL commands for generating the certificate.  Andy Echols figured this out for me, I don’t claim to have those skills.  The pcks12 format includes public and private keys for the certificate, and is easy to import and use in Windows.

1. openssl req -new -days 365 -nodes -keyout Finished/username-key.pem -out Meta/username-req.pem -config Meta/efs-fr.cnf
2. openssl ca -policy policy_anything -in Meta/username-req.pem -out Finished/username-crt.pem -extfile Meta/efs-fr.cnf
3. openssl pkcs12 -export -in Finished/username-crt.pem -inkey Finished/username-key.pem -certfile cacert.pem -out Finished/username.p12

The Meta/efs-fr.cnf config file is here.
The final product, after being imported into Windows, looks like this:

Final steps for creating a recovery agent

1. Import your cert

1. Log into a domain controller as an Enterprise Administrator (I’ll use FRuser for this example)

 

2. Import your File Recovery certificate

 

3. Verify the import worked and the certificate shows the File Recovery property in the Enhanced Key Usage (see image above).  Verification steps: start -> run -> mmc -> OK -> File -> Add/Remove Snap-in -> Add -> Certificates -> Add -> Close -> OK -> expand Certificates – Current User -> Personal -> Certificates -> double-click the name of the certificate you just imported -> click the Details tab -> select the Enhanced Key Usage line and verify “File Recovery” shows up (like the image above).

2. Assign your cert to your user

1. Open Active Directory Users and Computers
2. Open the properties for your user
3. Click the Published Certificates tab.  If you don’t see this tab.  Click View -> Advanced Features in the Active Directory Users and Computers window.
4. Click the Add from Store button
5. Select your File Recovery certificate and click the “View Certificate” button to make sure you selected the right one
6. Click OK in the Select Certificate window to add the cert to Published Certificates

3. Add yourself as a recovery agent for the domain

1. In Active Directory Users and Computers, right-click the domain (or a test OU) and select properties
2. Click the Group Policy tab and edit the Default Domain Policy
3. Drill down to the EFS folder (Computer Configuration -> Windows Settings -> Security Settings -> Public Key Policies -> Encrypting File System)
4. Right-click the Encrypting File System folder and select Add Data Recovery Agent…
5. In the wizard, click Next to skip the intro -> Browse Directory -> search for your username -> select your username from the results -> OK.  You will also have to click “Yes” to say you understand that “Windows cannot determine if this certificate has been revoked.”  This message is displayed because you are using a non-Microsoft CA.  Click Next -> Finish
6. You should now see your certificate in the GPO

4. Testing EFS & File Recovery

1. Log on to a domain workstation as a regular user (non-domain admin)
2. Create a new folder on your desktop called efs_test
3. Right-click the folder -> properties -> Advanced -> check the box “Encrypt contents to secure data” -> OK -> OK
4. If you get an error, you may need to run gpupdate on your workstation to make sure the GPO created in the previous section is effective
5. After encrypting the folder, create a new text file in the folder and type a few lines, save the file and log out.
6. Log into the workstation as the File Recovery user. 
7. Import your File Recovery certificate and private key.  This is critical!  The file was encrypted using public keys from the regular user and from the File Recovery user.  The file is decrypted using the private key, so that must be imported on the local machine for the FR user.
8. You should now be able to go into the user’s folder and read the encrypted file.
9. Running gpresult from the command line on the workstation can help troubleshoot GP issues, especially if you added a new GPO that conflicts with the Default Domain Policy.

5. Troubleshooting

If any comments to this post result in valueable troubleshooting steps, I’ll be sure to post those here.
A grahical illustration of how Windows encryption and file recovery works can be found here.