Circumventing ThinkPad’s WiFi Card Whitelisting

What started as a simple Mini PCI Express WiFi card swap on a ThinkPad T61 notebook, turned into deploying a custom BIOS in order to get the card to work.

I love ThinkPad notebooks, they are workhorses that keep on going and going. I always keep my older models around for testing, and one of my old T61’s had an Intel 4965AGN card, that worked fine with Windows 10, until the release of the Anniversary / Redstone 1 update. After the RS1 update, WiFi would either fail to connect, or randomly drop out. The 4965AGN card is not supported by Intel on Win10, and the internet is full of problem reports of Win10 and 4965AGN cards.

Ok, no problem, I’ll just get a cheap, reasonably new, with support for Win10, Mini PCIe WiFi card, and swap the card. I got an Intel 3160 dual band 802.11AC card and mounting bracket for about $20. The 3160 is a circa 2013 card with Win10 support. I installed the card, booted, and got a BIOS error 1802: Unauthorized network card is plugged in.

This lead me to the discovery of ThinkPad hardware whitelisting, where the BIOS only allows specific cards to be used, which lead me to Middleton’s BIOS, a custom T61 BIOS, that removes the hardware whitelisting, and enables SATA-2 support. I found working download links to the v2.29-1.08 Middleton BIOS here.

The BIOS update is packaged as a Win7 x86 executable or DOS bootable ISO image. As I’m running Win10 x64, and I could not find any CD-R discs around, I used Rufus to create a bootable DOS USB key, and I extracted the ISO contents using 7-Zip to a directory on the USB key. The ISO is created using a bootable 1.44MB DOS floppy image, and AUTOEXEC.BAT launches “FLASH2.EXE /U”, I created a batch file that does the same.

I removed the WiFi card, booted from USB, ran the flash, and got an error 1, complaining that flashing over the LAN is disabled. Ok, I enabled flashing the BIOS over the LAN in the BIOS, and rebooted.

I ran the update again, and this time I got error 99, complaining that BitLocker is enabled, and to temporarily disable BitLocker. I did not have BitLocker enabled, so I removed the hard drive and tried again, same error. Must be something in the BIOS, I disabled the security chip in the BIOS, tried again, and the update starts, but a minute or so later the screen goes crazy with INVALID OPCODE messages.

Hmm, maybe the updater does not like the FreeDOS boot image used by Rufus. Ok, let me create a MS-DOS USB key, uhh, on Win10, that turned out to be near impossible. Win10 does not include MS-DOS files, Rufus does not support custom locations for MS-DOS files, nor does it support getting them from floppy or CD images (readily available for download), the HP USB Disk utility complains my USB drive is locked, and writing raw images to USB result in a FAT12 disk structure that is too small to use. I say near impossible because I gave up, and instead went looking for an existing MS-DOS USB key I had made a long time ago. I am sure with a bit more persistence I could have found a way to create MS-DOS bootable USB keys on Win10, but that is an exercise of another day.

Trying again with a MS-DOS USB key, and voilà, BIOS flashed, and WiFi working.

I am annoyed that I had to go to this much trouble to get the new WiFi card working, but the best part of the exercise turns out to be the SATA-2 speed increase. This machine had a SSD drive, that I always found to be slow, but with the SATA-2 speed bump in Middleton’s BIOS, the machine is noticeably snappier.

A couple hours later, my curiosity got the better of me, and I made my own version of Rufus that will allow formatting of MS-DOS USB drives on Win10. In the process I engaged in an interesting discussion with the author of Rufus. I say interesting, but it was rather frustrating, Microsoft removed the MS-DOS files from Win10, and Rufus refuses to add support for sourcing of MS-DOS files from a user specified location, citing legal reasons, and my reluctance to first report the issue to FreeDOS. Anyway, can code, have compiler, if have time, will solve problem.

Advertisements

Self Signed BitFury Drivers

Almost two years ago I pre-ordered some bitcoin mining hardware from Butterfly Labs, what a waste. After countless delays, more than a year late, they finally shipped the hardware, and given the low probability of ever recovering the money through mining, I immediately sold the hardware on eBay, for a little profit.

In the mean time USB stick miners became available, outperforming GPU mining, and easy to setup and run. I’ve had a couple of ASICMiner Block Erupter’s running under my desk for some time, in the early days I saw some fractions of coins coming in, but in recent months they are so under-powered against the current hash-rates that they do little more than blink lights.

There is a resurgence in USB stick mining hardware, specifically the Bitfury type devices, many based on the NanoFury open source project that provided software, design, and PCB schematics.

I got myself a Red Fury, an Nano Fury II, and a Hex Fury. Compared to the 300MH/s of my little Block Erupters, these run at 2GH/s, 4GH/s, and 11GH/s respectively. There is still no way to ever make a profit in mining (at this scale), but I was really interested in seeing how these newer generation devices worked, especially since the publication of the NanoFury open source project, where in theory I could build my own.

So what does this have to do with self signed drivers, well, my mining tool of choice is CGMiner, but CGMiner currently only runs Nano Fury II’s at half speed, requiring the use of BFGMiner to go full speed. But unlike CGMiner that accesses all USB devices via Zadig installed WinUSB drivers, BFGMiner requires native Windows drivers, and neither the Red Fury nor the Hex Fury drivers are signed, so no installation on Windows 8 x64 (without disabling driver signing on every boot).

Looking at the INF files, all these devices do is register the USB hardware id as a generic null modem USB to COM bridge device, so no binaries required, just a signed CAT file.

After a bit of searching I found that I was not alone in my frustration, and I found a self-signed Red Fury driver. But, the Hex Fury used a different hardware id, and most people used CGMiner, so no need for a signed native driver as Zadig took care of that for us. So, I created my own signing script and signed my own drivers, install ok, BFGMiner happy.

If you just want signed drivers, get a copy of self-signed “Bitfury BF1” and “bi•fury” drivers here.

If you are interested in signing your own drivers, read on.

I tested on Windows 8.1 Update 1 x64:

Install the Windows 8.1 SDK and WDK.
Get the original “Bitfury BF1” and “bi•fury” INF files.
The bf1.inf file is saved in *NIX format (CR), convert it to Windows format (CRLF).
Create a self signed certificate:


makecert.exe -r -pe -ss PrivateCertStore -sr localMachine -n "CN=BitFury Test Signing Certificate" "C:\BitFury\BitFuryTest.cer"

Self Signed Certificate

Prepare the INF files, and create CAT files:


stampinf.exe -n -f "C:\BitFury\bf1.inf" -d * -v * -c "bf1.cat"
stampinf.exe -n -f "C:\BitFury\bifury_c4C.inf" -d * -v * -c "bifury_c4C.cat"
inf2cat.exe /v /driver:C:\BitFury\ /os:7_x86,7_x64,8_x86,8_x64

Sign the CAT files:


signtool.exe sign /v /s PrivateCertStore /n "BitFury Test Signing Certificate" /t http://timestamp.verisign.com/scripts/timestamp.dll "C:\BitFury\bf1.cat"
signtool.exe sign /v /s PrivateCertStore /n "BitFury Test Signing Certificate" /t http://timestamp.verisign.com/scripts/timestamp.dll "C:\BitFury\bifury_c4C.cat"

To use the drivers, you have to import the signing certificate into the local certificate store. As this is basically a self-signed CAT file, there are no trusted root certificates in the system that signed the signing certificate, and we need to add the signing certificate to the root and the trusted certificate stores.


certmgr.exe -add -c "C:\BitFury\BitFuryTest.cer" -s -r localMachine root
certmgr.exe -add -c "C:\BitFury\BitFuryTest.cer" -s -r localMachine trustedpublisher

Alternatively you can run “certlm.msc”, and import the certificate file into the “Trusted Root Certification Authorities” and the “Trusted Publishers” hives.

SelfSign.2 SelfSign.1

Last thing left to do is to use your newly signed drivers when selecting the custom driver from device manager.

SelfSign.4

Here is a package with signed drivers, and scripts to help you sign your own INF files, and import the certificate. It should work on Windows 7 and Windows 8 x86 and x64. Your mileage may vary, use at your own risk 🙂

Windows 8 VIDEO_TDR_FAILURE Madness

I finally figured out why I kept on getting VIDEO_TDR_FAILURE BSOD’s when installing Windows 8 on my SuperMicro workstations. It turns out that the problem goes away when I use a PCIe slot associated with CPU #1, instead of a slot associated with CPU #2.

Some history on my adventures with Windows 8 and SuperMicro SuperWorkstations:
I got ACPI_BIOS_ERROR BSOD’s while installing Windows 8, SuperMicro provided a Beta BIOS that resolved the problem.
The Windows 8 install hangs if installing to a SSD drive on a LSI 2308 SAS controller, that issue is still unresolved, but can be worked around by connecting the SSD to the Intel SATA controller.
I got VIDEO_TDR_ERROR BSOD’s while installing Windows 8 with a NVidia Quadro 5000 graphic card, same with an ATI FirePro V7900 or a NVidia GeForce GTX 680 or an ATI HD 7970. And this post is about resolving that problem.

 

SuperMicro released v1.0a BIOS updates for the X9DAi and X9DA7 motherboards used in the 7470A-T and 7470A-73 SuperWorkstations. I was hoping this will resolve the VIDEO_TDR_FAILURE BSOD’s, but no.

The X9DA7 BIOS updated without issue, but the X9DAi update reported an error at the end of the update process; “Error when sending Enable Message to ME”.

I contacted SuperMicro support, and they asked me to make sure that there is no jumper on JPME1. There is no mention of JPME1 in the motherboard manual, but it is located next to JIPMB1, next to PCIe slot #1. The header had a jumper on pins 2 and 3, where the same header on the X9DA7 motherboard had a jumper between 1 and 2. I removed the jumper, and the BIOS update succeeded.

JPME1

 

Unlike the ACPI_BIOS_ERROR BSOD that happens during the WinPE phase of the install, the VIDEO_TDR_FAILURE BSOD happens on the first boot after the install, during the hardware detection and driver install phase. This means that the technique I used to kernel debug the initial boot phase will not work, as the second boot is using the BCD already deployed to the target hard drive. I had to modify the BCD of the already installed image, prior to the install continuing after the reboot.

 

I tested many permutations of graphic cards and configurations, and it quickly became very annoying to have to type my Win8 product key every single time I boot and install. To avoid this I created configuration files in the sources directory on the install media, and this bypassed the key question. You can read more about the meaning of the file contents here:

EI.cfg:

[EditionID]
Professional
[Channel]
Retail
[VL]
0

PID.txt:

[PID]
Value=XXXXX-XXXXX-XXXXX-XXXXX-XXXXX

 

To modify the BCD of the installed image, and be able to easily repeat the second phase of install testing, I installed a second hard drive, and deployed WinPE to the second drive. By using F11 during boot to choose the boot drive, I could select booting from the second drive at any time.

 

I have a variety WinPE v3 (Win7) based utility images, and I updated them to use WinPE v4 (Win8). In the process I lost the boot menu, and the first image in the menu automatically started booting. After some trial and error, I found the bootmenupolicy BCD option, and when set to legacy mode, the old style menu is back:

bcdedit /set {default} bootmenupolicy legacy

 

I installed Win8 on the primary drive, and during the reboot, instead of booting to the installed Win8 drive, I used F11 and booted to my secondary WinPE drive. From WinPE I modified the boot BCD to enable kernel debugging over the network:

bcdedit -store c:\boot\bcd /set {default} nocrashautoreboot yes
bcdedit -store c:\boot\bcd /set {default} debugtype net
bcdedit -store c:\boot\bcd /set {default} hostip 3232235876
bcdedit -store c:\boot\bcd /set {default} port 50000
bcdedit -store c:\boot\bcd /set {default} key my.secret.debug.key
bcdedit -store c:\boot\bcd /debug {default} yes

This is equivalent to:

bcdedit /dbgsettings net host:192.168.1.100 port:50000 key:my.secret.debug.key

But unlike the dbgsettings command, this allows me to specify a BCD store. Also note that the IP address is stored as a single numeric value instead of the dotted IP format.

 

While still in WinPE, I captured the state of the primary Win8 drive by making a drive image using Symantec Ghost, the real Ghost, currently sold as Symantec Ghost Solution Suite, not the same named but volume snapshot based Norton Ghost or Symantec System Recovery. By saving a drive image, I can easily change hardware or configurations, test the install starting at the second phase, reboot to the secondary WinPE drive using F11, restore the entire drive image, and try again, while leaving the kernel debug options intact.

 

I tested with following hardware configurations in various permutations:

 

With the kernel debugger attached, I captured the following crash details in WinDbg for NVidia based cards:

VIDEO_TDR_FAILURE (116)
Attempt to reset the display driver and recover from timeout failed.
Arguments:
Arg1: fffffa80211cd010, Optional pointer to internal TDR recovery context (TDR_RECOVERY_CONTEXT).
Arg2: fffff8800782d0d8, The pointer into responsible device driver module (e.g. owner tag).
Arg3: 0000000000000000, Optional error code (NTSTATUS) of the last failed operation.
Arg4: 0000000000000002, Optional internal context dependent data.

Debugging Details:
------------------

FAULTING_IP:
nvlddmkm+1ae0d8
fffff880`0782d0d8 4055 push rbp

DEFAULT_BUCKET_ID: GRAPHICS_DRIVER_TDR_FAULT

BUGCHECK_STR: 0x116

PROCESS_NAME: System

CURRENT_IRQL: 0

STACK_TEXT:
fffff880`12c76078 fffff801`66fef0ea : 00000000`00000000 00000000`00000116 fffff880`12c761e0 fffff801`66f734b8 : nt!DbgBreakPointWithStatus
fffff880`12c76080 fffff801`66fee742 : 00000000`00000003 fffff880`12c761e0 fffff801`66f73e90 00000000`00000116 : nt!KiBugCheckDebugBreak+0x12
fffff880`12c760e0 fffff801`66ef4144 : fffffa80`2094b100 fffff880`021ee9c0 fffffa80`1f54e400 00000000`00000000 : nt!KeBugCheck2+0x79f
fffff880`12c76800 fffff880`04b33dcb : 00000000`00000116 fffffa80`211cd010 fffff880`0782d0d8 00000000`00000000 : nt!KeBugCheckEx+0x104
fffff880`12c76840 fffff880`04b32518 : fffff880`0782d0d8 fffffa80`211cd010 fffff880`12c76949 00000000`000000c7 : dxgkrnl!TdrBugcheckOnTimeout+0xef
fffff880`12c76880 fffff880`04a1e608 : fffffa80`211cd010 fffff880`12c76949 00000000`00000000 00000000`00000002 : dxgkrnl!TdrIsRecoveryRequired+0x168
fffff880`12c768b0 fffff880`04a4d539 : 00000000`00000000 fffff780`00000320 00000000`00000000 fffffa80`1f54e400 : dxgmms1!VidSchiReportHwHang+0x438
fffff880`12c769b0 fffff880`04a4ba49 : fffffa80`00000002 fffffa80`1f54e400 fffffa80`1f54e840 fffffa80`1f54e840 : dxgmms1!VidSchiCheckHwProgress+0xe5
fffff880`12c76a00 fffff880`04a16fe5 : ffffffff`ff676980 00000000`00000001 fffff880`12c76b69 fffffa80`1f54e400 : dxgmms1!VidSchiWaitForSchedulerEvents+0x20d
fffff880`12c76aa0 fffff880`04a4b646 : 00000000`00000000 00000000`0000000f fffffa80`1f54e400 fffffa80`1f54e400 : dxgmms1!VidSchiScheduleCommandToRun+0x289
fffff880`12c76bd0 fffff801`66e9b521 : fffffa80`1f5abb00 fffffa80`1f54e400 fffff880`03b01140 00000000`06a21e1e : dxgmms1!VidSchiWorkerThread+0xca
fffff880`12c76c10 fffff801`66ed9dd6 : fffff880`03af5180 fffffa80`1f5abb00 fffff880`03b01140 fffffa80`19aac040 : nt!PspSystemThreadStartup+0x59
fffff880`12c76c60 00000000`00000000 : fffff880`12c77000 fffff880`12c71000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16

STACK_COMMAND: .bugcheck ; kb

FOLLOWUP_IP:
nvlddmkm+1ae0d8
fffff880`0782d0d8 4055 push rbp

SYMBOL_NAME: nvlddmkm+1ae0d8

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nvlddmkm

IMAGE_NAME: nvlddmkm.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 4fdf93d7

FAILURE_BUCKET_ID: 0x116_IMAGE_nvlddmkm.sys

BUCKET_ID: 0x116_IMAGE_nvlddmkm.sys

 

With the kernel debugger attached, I captured the following crash details in WinDbg for ATI based cards:

VIDEO_TDR_FAILURE (116)
Attempt to reset the display driver and recover from timeout failed.
Arguments:
Arg1: fffffa801ed114d0, Optional pointer to internal TDR recovery context (TDR_RECOVERY_CONTEXT).
Arg2: fffff8800725cefc, The pointer into responsible device driver module (e.g. owner tag).
Arg3: 0000000000000000, Optional error code (NTSTATUS) of the last failed operation.
Arg4: 000000000000000d, Optional internal context dependent data.

Debugging Details:
------------------

FAULTING_IP:
atikmpag+8efc
fffff880`0725cefc 4055 push rbp

DEFAULT_BUCKET_ID: GRAPHICS_DRIVER_TDR_FAULT

BUGCHECK_STR: 0x116

PROCESS_NAME: System

CURRENT_IRQL: 0

STACK_TEXT:
fffff880`06fa9ee8 fffff803`e6ff20ea : 00000000`00000000 00000000`00000116 fffff880`06faa050 fffff803`e6f764b8 : nt!DbgBreakPointWithStatus
fffff880`06fa9ef0 fffff803`e6ff1742 : 00000000`00000003 fffff880`06faa050 fffff803`e6f76e90 00000000`00000116 : nt!KiBugCheckDebugBreak+0x12
fffff880`06fa9f50 fffff803`e6ef7144 : fffffa80`1e2df4e0 fffff880`020b99c0 fffffa80`1d31f010 00000000`00000000 : nt!KeBugCheck2+0x79f
fffff880`06faa670 fffff880`04d31dcb : 00000000`00000116 fffffa80`1ed114d0 fffff880`0725cefc 00000000`00000000 : nt!KeBugCheckEx+0x104
fffff880`06faa6b0 fffff880`04d30548 : fffff880`0725cefc fffffa80`1ed114d0 fffff880`06faa7b9 00000000`00000180 : dxgkrnl!TdrBugcheckOnTimeout+0xef
fffff880`06faa6f0 fffff880`04c11608 : fffffa80`1ed114d0 fffff880`06faa7b9 00000000`0000000f fffffa80`1d31f8f8 : dxgkrnl!TdrIsRecoveryRequired+0x198
fffff880`06faa720 fffff880`04c459f9 : 00000000`00000001 fffff880`06faa8a0 fffff880`06faa920 00000000`00000000 : dxgmms1!VidSchiReportHwHang+0x438
fffff880`06faa820 fffff880`04c3ff72 : fffffa80`1d31f010 fffff780`00000320 fffffa80`1d31f770 fffffa80`1d31f010 : dxgmms1!VidSchWaitForCompletionEvent+0x411
fffff880`06faa8e0 fffff880`04c4206c : fffffa80`1d31f010 fffffa80`1d31f450 fffffa80`1d31f450 00000000`00000000 : dxgmms1!VidSchiWaitForEmptyHwQueue+0x9a
fffff880`06faa9d0 fffff880`04c3ea85 : 00000000`00000000 fffffa80`1d31f010 fffffa80`1d31f450 00000000`00000000 : dxgmms1!VidSchiSuspend+0x74
fffff880`06faaa00 fffff880`04c09fe5 : ffffffff`ff676980 00000000`00000001 fffff880`06faab69 fffffa80`1d31f010 : dxgmms1!VidSchiWaitForSchedulerEvents+0x249
fffff880`06faaaa0 fffff880`04c3e646 : 00000000`00000000 fffffa80`1d585660 fffffa80`1d44d7f0 fffffa80`1d31f010 : dxgmms1!VidSchiScheduleCommandToRun+0x289
fffff880`06faabd0 fffff803`e6e9e521 : fffffa80`1d6b9b00 fffffa80`1d31f010 fffff880`03932140 00000000`04d91ecb : dxgmms1!VidSchiWorkerThread+0xca
fffff880`06faac10 fffff803`e6edcdd6 : fffff880`03926180 fffffa80`1d6b9b00 fffff880`03932140 fffffa80`19ac7500 : nt!PspSystemThreadStartup+0x59
fffff880`06faac60 00000000`00000000 : fffff880`06fab000 fffff880`06fa5000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16

STACK_COMMAND: .bugcheck ; kb

FOLLOWUP_IP:
atikmpag+8efc
fffff880`0725cefc 4055 push rbp

SYMBOL_NAME: atikmpag+8efc

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: atikmpag

IMAGE_NAME: atikmpag.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 4fdf9279

FAILURE_BUCKET_ID: 0x116_IMAGE_atikmpag.sys

BUCKET_ID: 0x116_IMAGE_atikmpag.sys

 

This was not really helping me much, and I decided to repeat the tests but use the checked build of Windows 8 to help troubleshoot.

With the kernel debugger attached, I captured the following ASSERT during the boot:

Windows 8 Kernel Version 9200 MP (1 procs) Checked x64
Built by: 9200.16384.amd64chk.win8_rtm.120725-1247
Machine Name:
Kernel base = 0xfffff802`0e01d000 PsLoadedModuleList = 0xfffff802`0e760ac0
System Uptime: 0 days 0:00:06.228 (checked kernels begin at 49 days)
Assertion: The BIOS has reported inconsistent resources (_CRS). Please upgrade your BIOS.
ACPI!PnpBiosGetDeviceResourceList+0x15e:
fffff880`012c3c2a cd2c int 2Ch
...
Unknown bugcheck code (0)
Unknown bugcheck description
Arguments:
Arg1: 0000000000000000
Arg2: 0000000000000000
Arg3: 0000000000000000
Arg4: 0000000000000000

Debugging Details:
------------------

PROCESS_NAME: System

FAULTING_IP:
ACPI!PnpBiosGetDeviceResourceList+15e
fffff880`012c3c2a cd2c int 2Ch

ERROR_CODE: (NTSTATUS) 0xc0000420 - An assertion failure has occurred.

EXCEPTION_CODE: (NTSTATUS) 0xc0000420 - An assertion failure has occurred.

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

BUGCHECK_STR: 0x0

CURRENT_IRQL: 0

LOCK_ADDRESS: fffff8020e7c5d60 -- (!locks fffff8020e7c5d60)

Resource @ nt!PiEngineLock (0xfffff8020e7c5d60) Exclusively owned
Threads: fffffa8019a36040-01<*>
1 total locks, 1 locks currently held

PNP_TRIAGE:
Lock address : 0xfffff8020e7c5d60
Thread Count : 1
Thread address: 0xfffffa8019a36040
Thread wait : 0x105eccd4

LAST_CONTROL_TRANSFER: from fffff880012b736f to fffff880012c3c2a

STACK_TEXT:
fffff880`009b4b30 fffff880`012b736f : fffffa80`23a9e900 fffff880`012a7e01 fffff880`009b4c08 fffff880`012a7e70 : ACPI!PnpBiosGetDeviceResourceList+0x15e
fffff880`009b4bd0 fffff880`0125acba : fffffa80`23a9e900 fffffa80`19ac54c0 fffff880`012a7e70 fffffa80`1f477010 : ACPI!ACPIBusIrpQueryResourceRequirements+0x8b
fffff880`009b4c50 fffff802`0e91b6a4 : fffffa80`23a9e900 fffffa80`19ac54c0 fffff880`009b4db0 fffffa80`23a9e900 : ACPI!ACPIDispatchIrp+0x2a6
fffff880`009b4cf0 fffff802`0e91cd1b : fffffa80`23a9e900 fffff880`009b4db0 00000001`c00000bb 00000000`00000000 : nt!IopSynchronousCall+0x10c
fffff880`009b4d80 fffff802`0e915bdb : fffffa80`23a9e900 fffff880`009b4e50 fffffa80`23a4f850 00000000`0000001e : nt!PpIrpQueryResourceRequirements+0x5f
fffff880`009b4e10 fffff802`0e91748d : fffffa80`23a9b8e0 00000000`00000000 ffffffff`80000218 fffffa80`23a9b8e0 : nt!PiQueryResourceRequirements+0x47
fffff880`009b4ea0 fffff802`0e91a1f2 : fffffa80`23a9b8e0 fffffa80`23a9b8e0 00000000`00000001 00000000`00000000 : nt!PiProcessNewDeviceNode+0x159d
fffff880`009b5070 fffff802`0e08feb5 : fffffa80`19adcd20 00000000`00000000 fffff880`009b5358 00000000`00000000 : nt!PipProcessDevNodeTree+0x1fe
fffff880`009b5310 fffff802`0e08fb59 : 00000000`00000000 00000000`00000000 00000000`00000000 fffffa80`37e19cc0 : nt!PnpDeviceActionWorker+0x345
fffff880`009b53d0 fffff802`0ed4010d : 00000000`00000000 fffff8a0`00000007 fffff8a0`00f08c00 00000000`00000000 : nt!PnpRequestDeviceAction+0x2ed
fffff880`009b5420 fffff802`0ed3b39d : fffff802`0d536800 fffff802`0e7c83c0 00000000`00000006 fffff802`0d536800 : nt!IopInitializeBootDrivers+0x905
fffff880`009b5650 fffff802`0ed2deb5 : fffff802`0d536800 00000000`00000000 fffff802`0d536800 fffff802`0d51ebf0 : nt!IoInitSystem+0xb5d
fffff880`009b59b0 fffff802`0e82d013 : fffff802`0d536800 fffffa80`19a36040 00000000`00000000 fffffa80`19ab3040 : nt!Phase1InitializationDiscard+0x1899
fffff880`009b5bc0 fffff802`0e1b289e : fffff802`0d536800 fffff802`0d536800 00000000`00000000 00000000`00000000 : nt!Phase1Initialization+0x13
fffff880`009b5bf0 fffff802`0e24ef96 : fffff802`0e82d000 fffff802`0d536800 fffff802`0e6c6180 00000000`f8ffffff : nt!PspSystemThreadStartup+0x1a2
fffff880`009b5c60 00000000`00000000 : fffff880`009b6000 fffff880`009b0000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16

STACK_COMMAND: kb

FOLLOWUP_IP:
ACPI!PnpBiosGetDeviceResourceList+15e
fffff880`012c3c2a cd2c int 2Ch

SYMBOL_STACK_INDEX: 0

SYMBOL_NAME: ACPI!PnpBiosGetDeviceResourceList+15e

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: ACPI

IMAGE_NAME: ACPI.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 50109dd0

BUCKET_ID_FUNC_OFFSET: 15e

FAILURE_BUCKET_ID: 0x0_ACPI!PnpBiosGetDeviceResourceList

BUCKET_ID: 0x0_ACPI!PnpBiosGetDeviceResourceList

 

This is interesting, the kernel ASSERT’s on a problem reported by the BIOS.

I contacted SuperMicro support, they said they will investigate the BIOS failure, and they suggested I try to use PCIe slot #3 instead of slot #5. The motherboard manual mentions that slots #1, #2, and #3 are to be used if CPU #1 is installed, and slots #4, #5, and #6 to be used only if CPU #2 is installed.

PCIe

I have both processors installed, so not using the more conveniently located slot #5 never came to mind. I moved the graphic card to CPU #1 slot #3, and voila, install succeeded and Windows 8 was up and running!

 

I repeated the checked build test with the graphic card in slot #3, and the same BIOS ASSERT error was reported, so the BIOS ASSERT seems to be unrelated to the ACPI_TDR_FAILURE error.

 

This was a very frustrating problem, and I still don’t understand the root cause, but I am happy to be able to finally switch both workstations to Windows 8.

WordPress.com 404 With Blogger Permalinks

Part of the research I did before migrating from Blogger to WordPress.com, was to make sure that current Blogger permalinks will resolve correctly once the old posts were imported into WordPress.com. At the time all seemed fine, but soon after migrating, I received alerts from Google Webmaster Tools that there is an increase in site errors, specifically 404 errors.

Some background: Permalinks are the URL’s that point directly to specific posts on the blog. These URL’s are known by search engines, are shared on forums, and are basically the static address of posts. Blogger and WordPress.com use different styles of permalinks. WordPress.com allows some customization of permalinks, but unlike WordPress.org, there is no support for custom plugins to handle rewrites for permalinks, 302’s or 404’s.

Although not documented anywhere, WordPress.com does support Blogger style permalinks, and will correctly redirect the Blogger style link to the WordPress.com style page. As an example, see the links below, one for Blogger and one for WordPress.com:

http://blogdotinsanegenius.blogspot.com/2012/06/looks-can-be-deceiving.html
https://blogdotinsanegenius.wordpress.com/2012/06/looks-can-be-deceiving

Search engines will know the link using the old blogger style URL, and both styles of links will correctly resolve to the current page:

https://blog.insanegenius.com/2012/06/19/looks-can-be-deceiving
https://blog.insanegenius.com/2012/06/looks-can-be-deceiving.html

So why is it that Google Webmaster Tools reported a suddenly spike in 404’s?

Google.404.1

By reviewing the links that report 404, I noticed that the permalink format of certain posts on WordPress.com was slightly different to the Blogger permalinks.

http://blogdotinsanegenius.blogspot.com/2009/10/hitachi-a7k2000-and-seagate-barracude.html
http://blogdotinsanegenius.blogspot.com/2010/05/zotac-xboxhd-id11-mkv-h264-video.html
http://blogdotinsanegenius.blogspot.com/2008/03/printing-from-network.html

https://blogdotinsanegenius.wordpress.com/2009/10/11/hitachi-ultrastar-and-seagate-barracude-lp-2tb-drives/
https://blogdotinsanegenius.wordpress.com/2010/05/28/zotac-xboxhd-id11-mkv-h-264-video-playback-performance/
https://blogdotinsanegenius.wordpress.com/2008/03/30/printing-from-the-network/

Notice the difference? Blogger appears to keep links short, and remove words like “the” and “and”.

I contacted WordPress.com support, and they provided a manual solution. They suggested that I modify the “slug” of each 404 post to match the Blogger style permalink.

Slug

This resolved the problem with the top 404’s, but I would have expected the Blogger import plugin to take care of this for me.

But, I soon received another alert email from Google Webmaster Tools, and this time the 404 posts looked a bit different.

Google.404.2

Notice that all the links contain parameters in the URL (I think these are old style Google Analytics parameters), and without the parameter the redirect works, but with any parameters the redirect fails.

https://blog.insanegenius.com/2009/09/western-digital-re4-gp-2tb-drive.html
https://blog.insanegenius.com/2009/09/western-digital-re4-gp-2tb-drive.html?m=1

I again contacted WordPress.com support, and I am still awaiting a resolution.

[Update: 9 August 2012]
Just got an email from WordPress.com support, the problem with parameters is fixed, thank you.

SuperMicro Beta BIOS supports Windows 8 and Server 2012

In a previous post I reported that my SuperMicro SuperWorkstation 7047A-T failed to install Windows 8 or Windows Server 2012 due to a ACPI_BIOS_ERROR. I contacted SuperMicro support, and I was informed that new BIOS releases are on their way that will support Windows 8 and Server 2012.

This morning I received an email from SuperMicro, with a new Beta BIOS for the X9DAi motherboard used in the 7047A-T. The new BIOS allowed me to install Windows 8 and Server 2012.

I used a DOS bootable USB key, and installed the new BIOS.

The 7047A-T has USB ports on the back and on the front of the case. The ports on the front are all USB3, and it is not possible to boot from these ports, at least I have not yet found a configuration that allows booting from USB3 ports. I tried using USB2 keys and, my newest Kingston DataTraveler HyperX 3.0 super fast USB3 keys, the BIOS does not list any boot devices in these USB3 ports. To boot from USB you have to plug the USB key in one of the rear USB2 ports.

The new BIOS version is “1.0 beta”, compilation date “7/23/2012”. The BIOS screen looks like the more modern AMI EFI BIOS’s I’ve seen in other devices, i.e. the thin font instead of the classic console font.

BIOS.Beta

I performed a “Restore Optimized Defaults”, and then went through the options to see what has changed and what is new.

The [Advanced] [Chipset Configuration] [North Bridge] [IOH Configuration] now sets all PCIe busses to GEN3, the old BIOS defaulted to GEN2.

The [Advanced] [SATA Configuration] now enabled hot plug on all ports, the old BIOS defaulted to hot plug disabled.

The [Advanced] [Boot Feature] ads a new power configuration item called “EuP”. This seems to be related to EU Directive 2005/32/EC:

EU Directive 2005/32/EC enacted by the European Union member countries dictates that after January 1, 2010, no computer or other energy using product (EuP) sold in the member countries may dissipate more than 1 Watt in the standby (S5) state.

I measured the power utilization, and the machine uses 2W when powered off, 140W at idle in Windows 8 desktop, and 7W while sleeping.

I updated my Windows 8 USB key to the latest build (I have access to), booted from the USB key, and installed Windows 8 without any major issues.

I had swapped the NVidia Quadro 4000 for a faster ATI FirePro V7900. The v1.0 BIOS worked fine with the Quadro 4000, but after installing the V7900, the screen powered on and Windows 7 started booting before I had a chance to see the BIOS screen. After installing the new Beta BIOS, the V7900 works as expected and I can see the BIOS screen during POST.

This is a note for ATI; please make sure your VGA driver install UI fits on a 640×480 display. When I swapped the Quadro 4000 for the V7900, and rebooted into Windows 7, I booted into a 640×480 16 color screen. Imagine my frustration trying to guess which button has focus when you can only see the top half of the ATI driver installer.

Windows 8 automatically installed drivers for the V7900.

The only driver Windows 8 did not automatically install is the C600 chipset SAS driver. I installed the Intel Rapid Storage Technology Enterprise (RSTe) drivers, and that solved that problem.

While running Windows 7 on this machine, and running the Windows Experience Index Assessment, the test would always crash. The same test in Windows 8 completed successfully.

Win8.EI

I found the 2D and 3D results to be disappointing, and I tried to replace the “ATI FirePro V (FireGL V) Graphics Adapter (Microsoft Corporation – WDDM v1.20)” driver with the ATI Windows 8 Consumer Preview driver. Although the release notes indicate that the V7900 is supported, the driver installation failed with an unsupported hardware error. I’ll have to wait for newer Windows 8 drivers from ATI to see if the test scores improve.

I’m quite happy that I can use my new machines with Windows 8.

I just wish SuperMicro solved the BIOS incompatibility problems long ago, after all, it has been almost two years since the Windows 8 pre-release program started, and almost a year since the release of the public developer preview.

Debugging Windows 8 Install BSOD

In my last post I described how to prevent Windows from automatically restarting when encountering a BSOD during the OS install process. This allowed me to see the  ACPI_BIOS_ERROR fault code while installing Windows 8 on my new SuperMicro workstation. The new Windows 8 BSOD page looks friendly, but no longer displays any error parameters other than the main fault code.

In order to get additional details of the crash, I had to hook up a kernel debugger to the machine. Windows 8 adds USB3 and TCPIP kernel debug support, and I will describe how I used the TCPIP network option to capture details of the crash.

 

First thing to do is prepare our tools, download the Windows 8 Debugging Tools for Windows package, and the Windows 8 Symbols.

Unfortunately the debugging tools are no longer available as a standalone download, and you need to install the SDK or WDK on a Windows 8 system in order to get them, but you can choose to only install the debugging tools. Once you installed the debugging tools on one machine, you can copy the MSI installers or the directory to any other machines, including Windows 7 systems. You will find the tools in the “C:\Program Files (x86)\Windows Kits\8.0\Debuggers” folder.

Microsoft is pretty good at publishing symbols for most released versions of their products to their public symbol server, but I prefer to extract the symbols to a working directory on my machine, or to upload the symbols to our internal symbol server. You can install the downloaded symbols MSI package directly, or use the following command to extract the symbols from the MSI file to a location on disk. Run an elevated (right click run as administrator) command prompt, and type:

msiexec /a [symbol msi file name] /qb targetdir="[output directory]"

 

Next we need to enable kernel network debugging in the BCD options. This needs to be done on a Windows 8 machine as the network debugging command is not supported in older versions of BCDEdit. I should also call out that network debugging support is required for hardware logo certification, but not all current adapters support it. Insert the bootable Windows 8 USB key, run an elevated command prompt, and type:

bcdedit –store [usb key drive]:\boot\bcd /dbgsettings net hostip:[IP of WinDbg machine] port:50000

BCDEdit will output the connection security key that is required by WinDbg.

 

Start WinDbg, and enable network kernel debugging, entering the port number and security key.

WinDbg.Network

 

Boot the target machine, you will see the target machine connecting to WinDbg:

Microsoft (R) Windows Debugger Version 6.2.8400.0 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Using NET for debugging
Opened WinSock 2.0
Waiting to reconnect...
Connected to target 192.168.1.106 on port 50000 on local IP 192.168.1.100.
Connected to Windows 8 8400 x64 target at (Fri Jul 20 11:07:21.583 2012 (UTC - 7:00)), ptr64 TRUE
Kernel Debugger connection established.

And then the ACPI_BIOS_ERROR crash:

25: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

ACPI_BIOS_ERROR (a5)
The ACPI Bios in the system is not fully compliant with the ACPI specification.
The first value indicates where the incompatibility lies:
This bug check covers a great variety of ACPI problems.  If a kernel debugger
is attached, use "!analyze -v".  This command will analyze the precise problem,
and display whatever information is most useful for debugging the specific
error.
Arguments:
Arg1: 0000000000000003, ACPI_FAILED_MUST_SUCCEED_METHOD
    ACPI tried to run a control method while creating device extensions
    to represent the ACPI namespace, but this control method failed.
Arg2: fffffa8019f2f288, The ACPI Object that was being run
Arg3: ffffffffc0000034, return value from the interpreter
Arg4: 00000000494e495f, Name of the control method (in ULONG format)

Debugging Details:
------------------

ACPI_OBJECT:  fffffa8019f2f288

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  0xA5

PROCESS_NAME:  System

CURRENT_IRQL:  0

LAST_CONTROL_TRANSFER:  from fffff803ca1e617a to fffff803ca0e5870

STACK_TEXT: 
fffff880`053eb418 fffff803`ca1e617a : 00000000`00000000 00000000`000000a5 fffff880`053eb580 fffff803`ca16b930 : nt!DbgBreakPointWithStatus
fffff880`053eb420 fffff803`ca1e57d2 : 00000000`00000003 00000000`494e495f fffff803`ca168810 00000000`000000a5 : nt!KiBugCheckDebugBreak+0x12
fffff880`053eb480 fffff803`ca0eb044 : 00000000`c0000034 fffff880`01038255 fffffa80`1a50fe78 00000000`c0000034 : nt!KeBugCheck2+0x79f
fffff880`053ebba0 fffff880`01043949 : 00000000`000000a5 00000000`00000003 fffffa80`19f2f288 ffffffff`c0000034 : nt!KeBugCheckEx+0x104
fffff880`053ebbe0 fffff880`0103bded : 00000000`00000000 00000000`00000000 00000000`00008004 00000000`c0000034 : ACPI!ACPIBuildCompleteMustSucceed+0x39
fffff880`053ebc20 fffff880`010346bd : fffffa80`1a500000 00000000`00008000 00000000`00000000 fffffa80`37e80000 : ACPI!AsyncCallBack+0x7f
fffff880`053ebc50 fffff880`01034f56 : fffffa80`1a500000 fffff880`01072be0 00000000`00000000 00000000`00000002 : ACPI!RunContext+0x141
fffff880`053ebc90 fffff880`010386e3 : fffffa80`19b1c3a0 00000000`00000000 00000000`00000000 fffffa80`19a35258 : ACPI!InsertReadyQueue+0xd6
fffff880`053ebcc0 fffff880`0103862a : fffff803`ca2eb490 fffff880`01072be0 00000000`00000000 00000000`546c6d41 : ACPI!RestartCtxtPassive+0x2f
fffff880`053ebcf0 fffff803`ca0cb181 : fffffa80`19e06b00 00000000`00000080 fffff880`04ac6540 00000000`00000000 : ACPI!ACPIWorkerThread+0xea
fffff880`053ebd50 fffff803`ca0dae26 : fffff880`04aba180 fffffa80`19e06b00 fffff880`04ac6540 fffffa80`19a8f940 : nt!PspSystemThreadStartup+0x59
fffff880`053ebda0 00000000`00000000 : fffff880`053ec000 fffff880`053e6000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16

STACK_COMMAND:  kb

FOLLOWUP_IP:
ACPI!ACPIBuildCompleteMustSucceed+39
fffff880`01043949 cc              int     3

SYMBOL_STACK_INDEX:  4

SYMBOL_NAME:  ACPI!ACPIBuildCompleteMustSucceed+39

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: ACPI

IMAGE_NAME:  ACPI.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  4fe6a2b1

BUCKET_ID_FUNC_OFFSET:  39

FAILURE_BUCKET_ID:  0xA5_ACPI!ACPIBuildCompleteMustSucceed

BUCKET_ID:  0xA5_ACPI!ACPIBuildCompleteMustSucceed

Followup: MachineOwner

 

Even with all the crash details, it still doesn’t really help me make progress, as it has been two days since I logged the support request with SuperMicro, and no response yet.

CrashPlan Memory Utilization

I’ve been using CrashPlan as an online backup solution for quite some time, and it works really well.

I like the fact that I can subscribe to the consumer plan, with almost 3.5TB of data backed up, and that the backup client installs on a server OS. Many of the other “unlimited” backup providers I tested have restrictions in place that makes such a setup impossible.

CrashPlan sends email notifications about backup status, and I noticed that something was wrong with the backup:
CrashPlan.Email

I logged onto the machine, opened the main UI, and after a few seconds the UI just closed. opened it again, same thing, after about 15s the UI closed.

My initial thoughts were that it is a crash, but on attaching a debugger, the exit call stack showed that the process was cleanly terminated after receiving a signal.

On looking at the NT eventlog I could see that the service was restarting about every 15s:

The CrashPlan Backup Service service entered the stopped state.
The CrashPlan Backup Service service entered the running state.
The CrashPlan Backup Service service entered the stopped state.
The CrashPlan Backup Service service entered the running state.
The CrashPlan Backup Service service entered the stopped state.
The CrashPlan Backup Service service entered the running state.

The service wasn’t crashing, it was externally being stopped and restarted. I looked in the CrashPlan directory, and I found several log files with a naming like restart_1342296082496.log. The contents of these files looked like this:

Sat 07/14/2012 13:01:22.53 : "C:\Program Files\CrashPlan\bin\restart.bat"
ECHO is off.
Sat 07/14/2012 13:01:22.53 : APP_BASE_NAME=CrashPlan
Sat 07/14/2012 13:01:22.53 : APP_DIR=C:\Program Files\CrashPlan
ECHO is off.
Sat 07/14/2012 13:01:22.53 : Stopping CrashPlanService
The CrashPlan Backup Service service is stopping.
The CrashPlan Backup Service service was stopped successfully.

Sat 07/14/2012 13:01:25.05 : Sleeing 15 seconds...

Pinging 127.0.0.1 with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Sat 07/14/2012 13:01:39.08 : Starting CrashPlanService

The CrashPlan Backup Service service was started successfully.

ECHO is off.
Sat 07/14/2012 13:01:39.13 : Exiting...

I looked for a newer version, but 3.2.1 was the latest version. I logged a support ticket with CrashPlan, but I continued my investigation. I found a log file service.log.0, several MB in size, and inside it I found this:

[07.14.12 12:32:39.480 ERROR   QPub-BackupMgr       backup42.service.backup.BackupController] OutOfMemoryError occurred...RESTARTING! message=OutOfMemoryError in BackupQueue!

So it seems that the service is running out of memory. I now had a few good keywords to search on, and I found this post of a user with the same problem. At about the same time I received a reply from CrashPlan support, not bad for weekend service, with the same solution.

The CrashPlan backup service and desktop applications are Java apps, and as such the maximum amount of memory they use are capped by configuration. I have had similar problems with other memory hungry Java apps, like Jaikoz, that simply fail unless you increase the memory limit.

To fix the problem, shutdown the service, open the CrashPlanService.ini file in the program directory, and increase the maximum memory utilization parameter to 2GB, the default is 512MB, and restart the service:

Virtual Machine Parameters=-Xrs -Xms15M –Xmx2048M

After upping the memory all seemed well, and the service has been running for more than a day. But, I wanted to know just how much memory is CrashPlan using, and it turns out to be insane.

Here are the current stats for the amount of data I backup, as well as the resource utilization by the backup service and desktop app:

CrashPlan.Size
CrashPlan.Memory.Desktop
CrashPlan.Memory.Service

As you can see, the desktop app’s peak private bytes exceed 250MB, and the service exceeds 1.3GB, that’s right 1.3GB of memory!

Those numbers are simply outrageous.

NCP VPN not quite Windows 7 compatible

I got so frustrated with the NCP VPN client software, specifically a recent update, that I decided to let my frustration out by writing about it.

 

For a long time I used the, IT approved, Cisco VPN client to access our corporate network. And as the OS technology moved along, so did my systems, meaning Vista, Windows 7, and 64-bit. While, for the most part, our IT approved systems remained to be only 32-bit XP. Cisco, for some reason, maybe they are representative of IT organizations as whole, was late to support Vista, was late to support Windows 7, and for the longest time never supported x64.

The only Cisco compatible alternative on Windows 7 x64, was the NCP VPN client. But, at a cost, $144, per machine, very expensive.

I recently read that almost 50% of Windows 7 machines are 64-bit, and I recently read that Cisco announced VPN client support for x64 on Vista and Windows 7. I am sure this is not a coincidence.

But I digress, this post is not about Cisco, it is about the NCP VPN client.

 

Where to start?

The NCP software is intrusive, every time you login it shows a splashscreen, and the splashscreen remains topmost, blocking anything behind it, until it closes, by itself. I contacted NCP support, complaining about the intrusion, and they told me that it is necessary to run their software at login so that they can validate the license. I replied that validating the license need not happen at every login, and certainly should not interfere with my system. They said I could search the internet and find out how other people disabled their software.

This means a few things to me; they do not value usability, they acknowledge there is a problem, yet they do not offer a solution.

Imagine if every application you install on your system decides it is a good idea to show a splashscreen when you login.

Here is the splashscreen that pops up on every login:
NCP.Splash

In case you are wondering how to disable NCP from running at startup, they install three user session startup entries under [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Run], delete or rename them

 

While we’re on the topic of usability, this application’s UI was probably not designed by Windows developers, or certainly not somebody that knows anything about standard Windows user interface design and principles.

In the main UI, shown below, how do you connect, where is the connect button, do you click the red button, no you need to click the gray area next to the red button. They probably though it looks cool.
NCP.UI.1

 

Every time you login the application starts and shows its UI. How would you normally look for options in a windows app; probably [Options], or [Tools][Options], or [File][Options]. No, you need to click on [View][Autostart][No Autostart], what does the [View] menu have to do with [Autostart]?
NCP.UI.7

And in case you were wondering, no, disabling autostart does not disable the splashcreen.

 

So what is it that made me mad enough to write this post?

I received an email that a new version was released, and the software offers a built in check for update mechanism by clicking [Help][Search for Updates]:
NCP.KB.vs.MB

How big is that update? 23.792 kByte, really, 24 kilobyte, or what is a lowercase k in kByte? Or is that really 23,729 KB as in 24 megabyte?
NCP.KB.vs.MB.1

Downloaded the update, now let’s apply it:
NCP.Reboot

And then the Windows Program Compatibility Assistant pops up:
NCP.AppCompat

 

I contacted support, got an email that said no other users had reported this problem, and they asked if I am an admin on the system, but before I could respond, I got another email that said they reproduced the problem, and that I should download and install the update from the website.

Let’s download the update directly, and install it:
NCP.Update

And then the Windows Program Compatibility Assistant pops up:
NCP.AppCompat.1

 

You may think it is a problem with my system, I repeated the same steps on a second system, that is how I captured the screenshots, and I ran Microsoft Process Monitor to record what is happening.

 

As a developer, that is familiar with writing Windows 7 compatible software, I know what is going on here.

The NCP software installs:
Three startup items: NcpBudgetGui, NcpPopup, NcpRsuGui
Three services: ncpclcfg, ncprwsnt, NcpSec
Two drivers: ncpfilt, ncplelhp

 

The NCP UI process is called NCPMON.exe, and is launched by explorer, runs non-elevated, at medium IL. It is the NCPMON.exe process that downloads and executes the update.

From the procmon log I can see that NCPMON.exe called CreateProcess() to launch the installer:
Date & Time:    7/21/2010 10:01:12 AM
Event Class:    Process
Operation:    Process Create
Result:    SUCCESS
Path:    C:\Users\Pieter Viljoen\AppData\Local\Temp\NCP_EntryCl_Win32_923_17.exe
TID:    6952
Duration:    0.0000000
PID:    6856
Command line:    "C:\Users\Pieter Viljoen\AppData\Local\Temp\NCP_EntryCl_Win32_923_17.exe"

 

This will not work, the NCP_EntryCl_Win32_923_17.exe is an installer it has to “Run As Admin” (this is an InstallShield installer, and although it does not contain a Run As Admin manifest entry, the Windows Application Compatibility subsystem recognizes InstallShield installers, and automatically runs them with elevation required), this means that when you launch this EXE using ShellExecute(), or using Explorer, you will get a UAC elevation prompt, and if approved, the installer will run elevated, and the install succeed.

There is only one explanation of how NCP could ever have shipped this, their developers and QA test with UAC disabled on their test systems.

 

What about the appcompat warning after the install?

Microsoft requires that Windows Vista and Windows 7 compatible applications mark their compatibility in the installer manifest.

By inspecting the resources in the installer EXE, there is a manifest, but there is no compatibility manifest, as such, this error will always show on a Windows 7 system.

I have no explanation for how NCP could allow this to ship, ignorance?

 

It makes me mad that an ISV advertises Windows 7 compatibility, and ships software like this. It is companies like NCP that gives Windows a bad name, and drives companies like Apple, to enforce rigorous, sometimes draconian, quality and usability standards in order to protect their own brand.

DELL 2408WFP loosing settings on power cycle

In my last post I discussed the calibration of my DELL 2408WFP monitors.

After calibration, and changing the monitor settings, the monitors looked pretty good, but I later found that the monitor colors looked all weird again.
It turns out that the monitors reverted to their default settings, invalidating the calibration.
It seems to me that as soon as my PC goes to sleep, or the monitors go into power saving mode, or powers off, that on turning back on they revert to default settings.
I found a relatively simple solution using EnTech mControl, not completely automated but close.
mControl allows you to save the current monitor settings to a profile, and allows you to restore those settings.
Here are the steps:
  1. Install and run mControl.
  2. Set mControl to automatically load when you login. Right click on the mControl tray icon and enable auto-load. This will add an entry in the startup program group.
  3. Calibrate your monitor, adjusting the monitor settings using mControl.
  4. Save your monitor profile. Open a command prompt, change to the mControl directory (“C:\Program Files (x86)\mControl\”), and run “mControl.exe /saveprofile Calibrate”. This will save the current monitor settings to a profile called “Calibrate”. You can use any profile name, I just used “Calibrate” as an example.
  5. Edit the mControl startup item so that it automatically restores the monitor profile when mControl starts. Right click the mControl entry in your startup programs group, and edit the commandline to include the “/restoreprofile Calibrate” option. E.g. “”C:\Program Files (x86)\mControl\mControl.exe” /restoreprofile Calibrate”
Every time you login mControl will start and restore the monitor settings.
If you change the monitor settings, simply run “mControl /saveprofile Calibrate” again to save the updated settings.
Unfortunately this only works when you login, but if the monitors power down while you are logged in, e.g. sleep, you have to manually restore the settings.
I solved this by creating a text script file called “Monitor.Restore.Profile.cmd” on my desktop, and putting the restore command in the file, “”C:\Program Files (x86)\mControl\mControl.exe” /restoreprofile Calibrate”.
Now whenever the monitor settings need to be fixed, I just run this script and the settings are restored.
This seems to be a problem with the DELL 2408WFP monitors, and I would like to know if this is specific my to my setup, or if this happens to other people, leave me a comment and let me know.

[Update: 17 July 2009]

EnTech has enhanced mControl to support profiles right in the UI, including an “Autoexec” profile that will automatically restore the monitor settings on login and wake from sleep.
It works great.

Amazon Unbox on x64 Vista

While shopping on Amazon I noticed that they were offering the Pilot of the Showtime series Nurse Jackie in HD for free, so I decided to give it a try.
The install (version 2.0.1.95) went smoothly, and the 1.3GB downloaded also completed pretty quickly, and I watched the show.
Ok, now I wanted to stop the Unbox player service from running and terminate the tray icon application, I have no need to have it running all the time.
I went to [Settings][Preferences], and unchecked the [Run the Amazon Unbox service when Windows starts] option.

I then right clicked on the tray icon and selected [Exit], the tray application launched “Amazon Unbox Config.exe stop” application, requiring UAC elevation, and promptly crashed with the following message:

An unhandled exception of type ‘System.BadImageFormatException’ occurred in Unknown Module.
Additional information: Could not load file or assembly ‘ADVWindowsClientAppRoot, Version=2.0.1.95, Culture=neutral, PublicKeyToken=091de1773ddefdbf’ or one of its dependencies. An attempt was made to load a program with an incorrect format.

I contacted Amazon support, and they provided this response:

Hello from Amazon.com.

I sincerely apologize for the trouble you’ve had using the Unbox Video Player. From your message, I understand you received an error relating to Windows being unable to load the correct file path.

I’ve researched the issue and suggest that you update the security components of your Microsoft operating system.

Please visit the Microsoft website listed below and follow Microsoft’s instructions for updating your security components. Microsoft may require you to use Internet Explorer to access all of the functions of this page and enable Active X controls in your Web browser.

http://drmlicense.one.microsoft.com/Indivsite/en/indivit.asp

If using Microsoft’s update does not resolve your playback issue, I recommend uninstalling and reinstalling your .NET Framework.

The Microsoft .NET Framework includes a large library of coded solutions to common programming problems and a virtual machine that manages the execution of programs written specifically for the framework. The .NET Framework is a key Microsoft offering and is intended to be used by most new applications created for the Windows platform.

I hope you found this information useful.

I gave them the benefit of the doubt and tried to update the DRM components, it did not work.
I suspected I know what the problem was, and this problem reminded me of a similar problem with the Google Email Uploader on Vista x64.
My suspicions were confirmed after I used CorFlags to inspect the binaries:

CorFlags.exe “C:\Program Files (x86)\Amazon\Amazon Unbox Video\Amazon Unbox Config.exe”
Microsoft (R) .NET Framework CorFlags Conversion Tool. Version 3.5.21022.8
Copyright (c) Microsoft Corporation. All rights reserved.

Version : v2.0.50727
CLR Header: 2.5
PE : PE32
CorFlags : 9
ILONLY : 1
32BIT : 0
Signed : 1

CorFlags.exe “C:\Program Files (x86)\Amazon\Amazon Unbox Video\ADVWindowsClientAppRoot.dll”
Microsoft (R) .NET Framework CorFlags Conversion Tool. Version 3.5.21022.8
Copyright (c) Microsoft Corporation. All rights reserved.

Version : v2.0.50727
CLR Header: 2.5
PE : PE32
CorFlags : 11
ILONLY : 1
32BIT : 1
Signed : 1

The output indicated that the EXE file was compiled to run natively on any platform, i.e. x64 on x64 and x86 on x86, but the DLL was compiled to be x86 only.
Thus the EXE runs as x64 and tries to load a x86 binary, not allowed, causing the crash.
The CorFlags output and x64 migration is discussed in this MSDN blog post:

anycpu: PE = PE32 and 32BIT = 0
x86: PE = PE32 and 32BIT = 1
64-bit: PE = PE32+ and 32BIT = 0

To fix the problem I have to change the EXE attributes to only run in 32bit:

CorFlags.exe “C:\Program Files (x86)\Amazon\Amazon Unbox Video\Amazon Unbox Config.exe” /32BIT+ /Force

The “Force” flag is required because the binary is Authenticode signed, and after the header change the Authenticode signature is now invalid.
“CorFlags” is parts of the .NET / Platform SDK and can be downloaded from Microsoft.

After I made the changes to the EXE, I repeated the original steps, and no more crash.
I replied to Amazon with my findings, and I hope they make the necessary, and easy, changes to fully support x64.