Circumventing ThinkPad’s WiFi Card Whitelisting

What started as a simple Mini PCI Express WiFi card swap on a ThinkPad T61 notebook, turned into deploying a custom BIOS in order to get the card to work.

I love ThinkPad notebooks, they are workhorses that keep on going and going. I always keep my older models around for testing, and one of my old T61’s had an Intel 4965AGN card, that worked fine with Windows 10, until the release of the Anniversary / Redstone 1 update. After the RS1 update, WiFi would either fail to connect, or randomly drop out. The 4965AGN card is not supported by Intel on Win10, and the internet is full of problem reports of Win10 and 4965AGN cards.

Ok, no problem, I’ll just get a cheap, reasonably new, with support for Win10, Mini PCIe WiFi card, and swap the card. I got an Intel 3160 dual band 802.11AC card and mounting bracket for about $20. The 3160 is a circa 2013 card with Win10 support. I installed the card, booted, and got a BIOS error 1802: Unauthorized network card is plugged in.

This lead me to the discovery of ThinkPad hardware whitelisting, where the BIOS only allows specific cards to be used, which lead me to Middleton’s BIOS, a custom T61 BIOS, that removes the hardware whitelisting, and enables SATA-2 support. I found working download links to the v2.29-1.08 Middleton BIOS here.

The BIOS update is packaged as a Win7 x86 executable or DOS bootable ISO image. As I’m running Win10 x64, and I could not find any CD-R discs around, I used Rufus to create a bootable DOS USB key, and I extracted the ISO contents using 7-Zip to a directory on the USB key. The ISO is created using a bootable 1.44MB DOS floppy image, and AUTOEXEC.BAT launches “FLASH2.EXE /U”, I created a batch file that does the same.

I removed the WiFi card, booted from USB, ran the flash, and got an error 1, complaining that flashing over the LAN is disabled. Ok, I enabled flashing the BIOS over the LAN in the BIOS, and rebooted.

I ran the update again, and this time I got error 99, complaining that BitLocker is enabled, and to temporarily disable BitLocker. I did not have BitLocker enabled, so I removed the hard drive and tried again, same error. Must be something in the BIOS, I disabled the security chip in the BIOS, tried again, and the update starts, but a minute or so later the screen goes crazy with INVALID OPCODE messages.

Hmm, maybe the updater does not like the FreeDOS boot image used by Rufus. Ok, let me create a MS-DOS USB key, uhh, on Win10, that turned out to be near impossible. Win10 does not include MS-DOS files, Rufus does not support custom locations for MS-DOS files, nor does it support getting them from floppy or CD images (readily available for download), the HP USB Disk utility complains my USB drive is locked, and writing raw images to USB result in a FAT12 disk structure that is too small to use. I say near impossible because I gave up, and instead went looking for an existing MS-DOS USB key I had made a long time ago. I am sure with a bit more persistence I could have found a way to create MS-DOS bootable USB keys on Win10, but that is an exercise of another day.

Trying again with a MS-DOS USB key, and voilà, BIOS flashed, and WiFi working.

I am annoyed that I had to go to this much trouble to get the new WiFi card working, but the best part of the exercise turns out to be the SATA-2 speed increase. This machine had a SSD drive, that I always found to be slow, but with the SATA-2 speed bump in Middleton’s BIOS, the machine is noticeably snappier.

A couple hours later, my curiosity got the better of me, and I made my own version of Rufus that will allow formatting of MS-DOS USB drives on Win10. In the process I engaged in an interesting discussion with the author of Rufus. I say interesting, but it was rather frustrating, Microsoft removed the MS-DOS files from Win10, and Rufus refuses to add support for sourcing of MS-DOS files from a user specified location, citing legal reasons, and my reluctance to first report the issue to FreeDOS. Anyway, can code, have compiler, if have time, will solve problem.

UPS Battery Replacement Turns Into Unrecoverable Firmware Update

Two lessons learned; do not trust scheduled battery tests, and leave working firmware be!

As the saying goes, if it is not broken do not fix it, especially when it comes to firmware.

I have a couple APC Smart-UPS‘s at my house, same as the models I like to use at the office. I use the SMT750 models with AP9631 Network Monitoring Cards. The problem started when we had a short power outage, and the UPS that powers the home network switch, cell repeater, alarm internet connection, and PoE IP cameras, unexpectedly died. A battery replacement led to the opportunity to do a UPS firmware update, which led to an unrecoverable firmware update.

It started when I woke up one morning and it was obvious the power had been out, first indicator is the kitchen appliances have blinking clocks, second are the numerous power failure email notifications, and the emails that stood out were from the alarm system that says it lost power and internet connectivity. The alarm has it’s own backup battery, the network switches and FiOS internet have their own battery backups, and the outage was only about 4 minutes.  So how is it that the UPS died, killing the switch, disconnecting the internet, especially when the outage was only 4 minutes, and typical runtimes on the UPS should be about an hour?

Here is the UPS outage log produced by the NMC card:

10/11/2016 06:53:05 Device UPS: A discharged battery condition no longer exists. 0x0108
10/11/2016 06:12:27 Device UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately. 0x0107
10/11/2016 06:12:24 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
10/11/2016 06:12:12 System Network service started. IPv6 address FE80::2C0:B7FF:FE98:9BAF assigned by link-local autoconfiguration. 0x0007
10/11/2016 06:12:10 Device Environment: Restored the local network management interface-to-integrated Environmental Monitor (Universal I/O at Port 1) communication. 0x0344
10/11/2016 06:12:09 System Network service started. System IP is from manually configured settings. 0x0007
10/11/2016 06:12:02 System Network Interface coldstarted. 0x0001
10/11/2016 05:45:36 Device UPS: A low battery condition no longer exists. 0x0110
10/11/2016 05:45:36 Device UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately. 0x0107
10/11/2016 05:45:35 Device UPS: The output power is turned off. 0x0114
10/11/2016 05:45:35 Device UPS: The graceful shutdown period has ended. 0x014F
10/11/2016 05:45:35 Device UPS: No longer on battery power. 0x010A
10/11/2016 05:45:35 Device UPS: Main outlet group, UPS Outlets, has been commanded to shutdown with on delay. 0x0174
10/11/2016 05:45:35 Device UPS: The power for the main outlet group, UPS Outlets, is now turned off. 0x0135
10/11/2016 05:45:18 Device UPS: The battery power is too low to continue to support the load; the UPS will shut down if input power does not return to normal soon. 0x010F
10/11/2016 05:41:36 Device UPS: On battery power in response to rapid change of input. 0x0109

I could see from the log that the UPS battery power ran out within 4 minutes, 05:41:36 on battery, 05:45:18 battery too low, 05:45:35 output turned off. The UPS status page was equally puzzling, load was at 9.7%, yet reported runtime was only 5 minutes, impossible.

Here is the status screenshot:


I ran a manual battery test, the test passed, but from the log it was clear the battery failed. I have bi-weekly scheduled battery tests for all UPS’s, never received a failure report. So what is the point of a battery test if the test comes back no problem yet it is clear to me from the logs that the battery failed?

Here is the log:

10/11/2016 18:07:06 Device UPS: A low battery condition no longer exists. 0x0110
10/11/2016 18:07:06 Device UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately. 0x0107
10/11/2016 18:07:05 Device UPS: Self-Test passed. 0x0105
10/11/2016 18:06:59 Device UPS: A discharged battery condition no longer exists. 0x0108
10/11/2016 18:06:59 Device UPS: The battery power is too low to continue to support the load; the UPS will shut down if input power does not return to normal soon. 0x010F
10/11/2016 18:06:58 Device UPS: Self-Test started by management device. 0x0137
10/11/2016 18:06:56 Device UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately. 0x0107

I asked for advice on the APC forum, no reply yet, and I ordered a replacement RBC48 battery. I received the battery, installed it, and the reported runtime is back to normal, 1 hour 48 minutes.

Here is a status screenshot with the new battery:


Here is where I should have stopped and called it a day, but no. I knew that the UPS’s were on old firmware, and I decided to use this opportunity to update the firmware. I’d normally let firmware be, unless I have a good reason to update, but I convinced myself that the new firmware readme had some fixes that may help with the false pass on the battery test:

Release Notes (UPS09.3):
2. Improved self-test logging for PCBE / NMC.
11. Repaired an occasional math error in the battery replacement date algorithm that resulted in incorrect dates.

I update the UPS, where I just replaced the battery, instructions are pretty simple. Only hassle is I have to bypass the network equipment to be mains powered so I can turn the UPS outputs off while updating the firmware, while maintaining network connectivity.

I did the same for my office UPS, PC and office switch on mains power, and when I power down the output, I made sure to not notify PowerChute Network Shutdown (PCNS) clients, as my PC had the PowerChute client installed to receive power state via the network. I start the firmware update over the network, and a few seconds later I get a Windows message that shutdown had been initiated by PCNS, what? I sit there in frustration, nothing to do but watch my PC shutdown while it is still delivering the firmware update.

On rebooting my PC, NMC comes up, but reports the UPS has stopped communicating. I pull AC power from the UPS, no change, I also pull the batteries, and when I plug the batteries and mains back on, beeeeeeeeeep. NMC now reports no UPS found, the UPS LCD panel reports all is fine. And still beeeeeeeeeep, and no way to stop the beeeeeeeeeep.

Here is the NMC status page:


I try to do Firmware Upgrade Wizard update via USB, plug a USB cable in, PC sees UPS, reports critical condition, but the upgrade wizard reports no UPS found on USB.

Here is the wizard error page:


So, here I am, stuck with a bricked UPS, lesson learned, actually two lessons learned; do not trust scheduled battery tests, and leave working firmware be!

Amazon Associate’s Account Closed

Amazon just notified me in email that my Associate’s account was closed due to not being in compliance with their operating agreement:

“You are not in compliance with Participation Requirement Number 29 because purchases resulting from Special Links on your site have been used for resale or commercial use.”

I have no idea how or why this happened.


A couple of years ago I moved my blog from the free Blogger platform to a paid hosted site. About the same time I signed up for an Amazon Associate’s account, profiting from any Amazon links resulting in sales, hoping that the proceeds would cover the costs of WordPress hosting and domain registration.

A quick calculation shows Amazon payouts of $669.57 between 2 August 2012 and 30 August 2016, that is about $167.39 per year, less the $99.00 for WordPress hosting, less $36.00 for Akismet blog spam filtering, less $19.00 for domain registration, leaves a profit of $13.39. Less $99.00 for bulk domain registration fees, not really fair to charge this fee to one domain, leaves a loss of $85.61 per year.


I do not know why I was suddenly out of compliance, I made no changes to either my Amazon Associates or WordPress accounts, and I’ve not posted any new content in a number of months. The WordPress stats show typical traffic (ignore the last two days), but the Amazon Associates report does show a marked increase in traffic:


I sent an email to Amazon support to clarify the violation, and to request my account be reinstated, but based on similar reports from other low traffic users,  I do not expect a resolution.

Instead, I opted-in to use WordPress’s own WordAds advertising platform, I still need to adjust the blog theme and settings to not interfere with reading, and I have no idea what the monetization would be, but at least I no longer have to bother with making special Amazon links.

Please comment and let me know if you find the ads to be intrusive, and I’ll consider funding the site without advertising assistance.


[Update: 1 September 2016]

A day after sending Amazon a request asking for an explanation, I received the following in email:

“This message is to advise you that your pievilsblo-20 account and your August 2016 Advertising Fees have been reinstated. Please accept our apologies for the closure.”

Looks like my account has been reinstated, no explanation of what happened.

Ecobee3 WiFi Connection Troubles

Ecobee3 thermostat WiFi troubles

In a previous post I wrote about my transition from Nest to Ecobee3 thermostats, and how the biggest benefit of the E3 was the use of remote sensors.

After several months of use, winter and summer, I find the remote sensors really do work very well, and our bedrooms remain at the desired temperature, while the areas around the thermostats can be warmer or colder.

But, the E3 is not perfect, there are two recurring problems; the remote sensors would report offline, and the units would lose network connectivity.


I’ve received sensor offline alerts a couple of times, typically happens early mornings, maybe interference, don’t know, the sensors never move from where they are placed.

Every time a I get a sensors offline report, the sensor already restored connectivity. This behavior is very annoying, I will get two emails a minute apart, and the E3 UI will have an alert saying sensor offline, I click ok, and then immediately an alert saying sensor online.


I expect the E3 to have some sort of grace period before it deems a problem so important that it needs to notify me. As is, it is just an annoyance as there is no remediation action to take.



The second problem is the E3 loses network connectivity, this is a real problem, as the units remain offline until power cycled, and to power cycle the E3 has to be removed from the wall bracket, i.e. there is no reboot menu option.

I reported this problem to Ecobee support in October, and on the SmartHomeHub community forum. Yes, it is a bit pathetic that Ecobee does not have their own support community forum. Ecobee had me reserve static IP’s in the DHCP server, setup a dedicated 2.4GHz SSID, still disconnects. By December the problem was still happening, and Ecobee support escalated the problem to their development team, it is two months later, and still no updates from Ecobee support on the problem.


Through my own research and experimentation I suspect the problem to be that the E3 is unable to handle a WiFi channel change, and unable to roam between access points. There are some conditions that trigger the problem that I cannot explain.

I have multiple access points in my house, same SSID, different channels, 2.4GHz and 5GHz bands. I tested with Ubiquity UniFi AC and with Ruckus/Xclaim Xi-3. And in case you’re wondering, no other devices in my house have any problems with WiFi, even with dynamic channel selection.

I can make the E3 fail by either changing the AP channel, or by making it roam to a different AP, also changing channels. If I configure the AP’s to use auto channel selection, then the E3 will fail as soon as the AP chooses to change channels (UniFi does this on startup, Xclaim does this dynamically). If I manually change the AP channel, the E3 will fail. If I take one AP offline, the E3 will fail to roam to a different AP (on a different channel).

Even with my AP’s configured with static non-overlapping channels, the E3 would still sometimes fail, requiring a power cycle. I do not know why this would happen, as signal strength by the E3’s are perfect.

On the plus side, the E3 units remember the schedule, and even when offline, they continue to operate.


Bottom line is E3 WiFi is not reliable, and E3 support/dev is not responsive.

Ecobee3 Thermostat with Remote Sensors

We’ve had a particularly warm summer, for our very moderate area, and between my wife and my parent’s in-law, they were constantly changing the thermostat temperature, leaving nobody particularly happy, and our electricity consumption skyhigh. I needed a better solution, I found one, but it has some quirks.

I was an early adopter of the Nest Generation 1 thermostat, and when we moved to our new house, Nest was still the best available, and I installed two Nest Generation 2 thermostats, one upstairs, and one downstairs.

Nest used to be an innovator and leader in the home thermostat space, and then two things happened; they were acquired by Google, and competitors like Ecobee and Honeywell released very competitive products. The just released Nest Generation 3 thermostat has no new notable features, it is simply thinner, not unlike its competitors.

The one feature I, and many other users, asked for was remote temperature sensors. In many homes, like ours, where there are two HVAC units, one for upstairs and one for downstairs, with no room specific dampers or temperature control, the upstairs and downstairs air mixes and causes large temperature differentials between closed rooms and open spaces. Adding to that warm air rises and cold air falls, so in the summer the upstairs pumps cold air downstairs, and in the winter the downstairs pumps warm air upstairs, this again leaves bedrooms too hot or too cold.

Ecobee solved this problem, to a large degree, with the Ecobee3 thermostat, that comes with one remote sensing unit, and extra sensors can be purchased at $35 per sensor. The latest version of their thermostat is also Apple HomeKit compatible, allowing Siri to control the thermostat.

There are 3rd party integrations that can control the Nest temperature, like Wally at an additional $299, or SmartThings at an additional $139, but these are integration solutions, not integrated solutions, and makes the Nest solution much more expensive, especially considering the Ecobee3 (with one extra sensor included) and the Nest Gen3 are both $249.

There are alternate solutions like EcoVent that controls the individual vents per room, but that adds an additional $499 minimum for two rooms.

The Ecobee3 solution cannot control the temperature in each room, for that you really need a split AC unit per room, but it does allow the temperature sensing logic to take input from any number of rooms. And in my case, I am specifically interested in the temperatures in the bedrooms, not the general open areas.

Replacing the Nest with an Ecobee3 was easy, the Ecobee3 is slightly thinner than the Nest Gen2, and slightly larger, with that extra size used for a multi-color touch display.

Size Comparison

The remote sensors come with stands, wall screw mounts, and wall sticky tape mounts, they are pretty small and unobtrusive.

Sensor Size

The WiFi setup was really easy, and the first time I’ve seen this particular scheme in action, I believe it is called Wireless Accessory Configuration (WAC), not sure, Apple documentation is as always in short supply. Basically it worked like this; install the EB3 app on my iPhone, EB3 told me to connect my phone to the EB3 SSID, my phone asked me if I want to connect the device to WiFi, select yes, and the EB3 was automatically connected to my home WiFi, no passwords, no hassles, easy.

For each thermostat install the EB3 app asked me details about my house, address, size, construction, etc. This was annoying, one house, one set of details, multiple thermostats, why do I need to configure this for every thermostat. The Nest config was always very easy, one set of options per house, multiple thermostats. A call to EB support told me I need to create a group, then add the thermostats to the same group, then select what options I want to share between thermostats in the same group, and this can only be done from the web portal. This was a setup and first experience usability fail.

The second problem I ran into was the time configuration, the EB3 correctly selected Los Angeles as the time zone, and synced the time, but the displayed time was 3 hours ahead, East Coast time instead of Pacific Time. I fixed this by manually changing the timezone to something other than LA, saving, then changing it back to LA, and the time was correct. I did send EB support an email about this, and they replied that I need to make sure I have the correct time zone selected, right.

Time Zone

Pairing the remote sensors was really easy, stand in front of the EB3 and pull the plastic tab to let the sensor battery make contact, the EB3 detects the sensor, and lets you pair it, and enter a sensor name. The EB3 thermostat UI allowed me to use non-alpha characters, e.g. “Childs Room”, note the apostrophe, but when I later renamed the sensors, the mobile app and the web app restricted names to numbers and letters only, a slight inconsistency fail.

Sensor Naming

Configuring the schedule and comfort zones was easy and intuitive. I particularly like the concept of the comfort zones, and reusing them in the schedule, vs. the Nest’s more primitive setting of desired temperatures at times of days. This is where I configured the night comfort zones to use only the sensors in the bedrooms, and ignore the temperature at the main thermostat. Made a huge difference in comfort and AC runtime.

After a couple days of use I noticed something weird, the AC would turn on, but the thermostats would show the current temperature is still below the set temperature. This happened with both thermostats, and only in the afternoon. The schedule only called for 74F after 8pm during the night time comfort zone, this was around 7pm, when the set temperature was 78F.

Something is clearly wrong with the scheduler and the schedule. My bet was the scheduler is still using East Coast time (turned out I was wrong), Ecobee phone support was already closed, so I had to wait for the following day.

While I was looking at the System Monitor feature, very neat, I noticed gaps in the data. After a bit of research I found that Ecobee is having scaling problems, and their backend cannot handle the load, may be exacerbated by the Apple store kicking out Nest and now selling Ecobee3, or HomeKit integration, or poor planning. It was also weird that Ecobee does not run their own support forums, the Ecobee community supports themselves at the SmartHomeHub forums. One intrepid forum user created an availability graph based on his data gaps, clearly shows the recent problems.

I called Ecobee support, and they explained what was going on with the schedule; the thermostat has a feature called Smart Recovery Mode, in this mode the AC starts running before a schedule change in an attempt to reach the desired temperature when the schedule starts. And that this prediction takes a week or so to become more accurate, and that it can be impacted by fluctuations in the weather. Ok, makes sense, but usability fail by not making this behavior clear in the status UI.

Smart Recovery

As for the gaps in data, Ecobee support said they are busy migrating data, that this impacts backend performance, and that all systems should be operational in a week, and no data should be lost, slight contradiction to the post on the SmartHomeHub forum, but at least acknowledged.

I am pretty happy with the E3, the remote sensors really do make a big difference in efficiency.


[Update: WiFi troubles with the E3]

MR16 12V Halogen to LED retrofit

This post is about my research into finding suitable MR16 LED’s for replacing the 50W recessed halogen lights in our house. In summary, I’ve found “ok” bulbs, not great bulbs, and you can read about the details below.

Our house is about 3 years old, new construction, and one of the many decisions we made during planning was recessed halogen vs. recessed LED lighting. At the time my calculations showed the additional cost for LED lights would only be recovered in electricity cost savings after about 12 years, not worth the cost at the time. Another problem was the optical quality of the products, the near halogen optical quality LED products were ridiculously expensive, and the mainstream LED’s were of poor optical quality, and had poor dimmability.

Given the situation we opted for recessed Elco MR16 low voltage 12V AC magnetic transformer halogens, and planned on retrofitting them with LED’s as the technology improved and costs came down.

Now, 3 years later; our electricity cost is way higher than originally estimated, we installed solar that gave us a 50% reduction in cost, some of the recessed reflectors are showing signs of heat damage from the halogen bulbs, and 12V MR16 LED’s have entered the mainstream.

I’ve been looking for MR16 LED’s for some time now, same problem as 3 years ago, dimmable good optical quality bulbs are very expensive, ~$20 per, while eBay and Amazon sourced Chinese manufactured no-name brands are ~$4.

During my research I’ve made a few important observations:

  • US electrical code requires the use of GU10 bi-pin twist-lock lamp bases for new construction, and GU10 will eventually replace all E26 style screw in bases. In the past months I found that there is a much wider supply of 110V GU10 base MR16 dimmable LED bulbs compared to GU5.3 12V bulbs. This is especially true for the no-name brand Chinese suppliers on eBay. I am assuming that the electronic circuitry used is similar to that used in the widely available regular E26 / A21 110V dimmable LED bulbs, and that the only difference is the MR16 housing construction. Keeping in mind that most installed AC dimmers are forward phase, and support a large variety of load types, while an LED is a constant current device that typically uses pulse width modulation for dimming. Thus a line voltage forward phase dimmer to pulse width modulated LED driver circuit is non-trivial, adding a 12V AC transformer in the mix, and supporting both 12V AC and 12V DC loads further complicates the circuitry, especially when dimming is required.
  • Just like there is an expanding variety of dimmable GU10 line voltage MR16’s, there is an equal growing number of line voltage dimmable retrofit LED housings, that are near the cost of a MR16 bulb. These housings replace the old recessed can with an integrated AC LED driver and LED bulb array.
  • Today, as was the case 3 years ago, commercial and residential LED recessed lights separate the enclosure from the LED driver, allowing for different color and brightness LED’s to be used, and to optimize the electronic circuitry for the type of dimmer being used. The prices are much lower compared to 3 years ago, but still high comparing a $40 halogen enclosure to a similar style $150 LED enclosure. It is possible to replace the entire enclosure, but it is a big job requiring ripping out ceiling drywall.
  • 12V MR16’s can be powered by 12V AC electromagnetic transformers or electronic low voltage drivers. ELV drivers offer much higher efficiencies, but require compatible dimmers, and sometimes dimmers specifically designed for ELV drivers. Halogens are almost always powered by electromagnetic transformers due to the reduced cost and complexity. It is possible to replace the electromagnetic transformers in the enclosure with an ELV transformer, I’ve seen the electrician replace a blown transformer, he had to bring in the “small hand guy” from his crew and even then it took a lot of blind finger fiddling.
  • I use a Vantage Controls InFusion home automation lighting control system in my house. The system supports line-voltage forward phase and reverse phase dimmer modules, 0-10V control, and PWM control LED dimming. All loads in my installation are dimmed using forward phase dimmers. The recommended LED control setup is to use the 0-10V or PWM circuits, i.e. the dimming control and power lines are separate. The 0-10V / PWM control modules are about the same cost per line as AC dimmer modules, but the real cost is again in LED driver circuitry.

So what are my realistic choices:

  • The best quality option is to replace the halogen housings and line voltage control circuitry with native PWM control and native LED drivers. But, same as during construction, this is not a cost effective solution.
  • I can remove the transformers and convert the enclosure to line voltage, and use GU10 type MR16’s. But, tricky to remove the transformer, and the safety and legal state of the enclosure would be unknown after being converted to a line voltage receptacle.
  • I can remove the transformers and convert the enclosure to line voltage as above, but instead of using MR16 form factor bulbs, I can replace the insert with LED retrofit inserts.
  • I can replace the electromagnetic transformers with ELV’s to improve efficiency and dimmability. But again, a tricky job with marginal cost savings, and I still end up using 12V LED MR16’s.
  • I’ve opted to replace the halogen bulbs with LED’s on an as needed basis, i.e. when I need to replace a burnt trim, or blown bulb, I will replace the entire zone of lights with the same model of LED’s. My hope is that there will be ongoing improvements in product quality and performance, and ongoing reduction in costs as availability goes mainstream.

Here then is my review, more an exploration, greatly skewed by my subjective opinion vs. scientific fact, of the products I could find and test.

I initially tested the bulbs by replacing the halogens in my office, but this quickly became cumbersome, so instead I created a test bed for evaluation, trying to simulate the various dimmer and transformer types available.

I bought the the following items to match what I use in my house:

Here are some pictures of the Elco enclosure, this will give you an idea of how to go about swapping the transformer, and how tight a squeeze it is:

The DV-603P is a vanilly halogen and incandescent dimmer, it works just fine with the magnetic transformer and halogen bulbs in my house, but the MR16 LED manufacturer’s compatibility guide require the use of specific low voltage magnetic or electronic low voltage dimmers. So I also bought:

I considered a more elaborate test setup, but I don’t have access to the required equipment, and the measurements would be interesting from a scientific perspective, not so much a subjective perspective. So I opted for a simpler test setup, attached to a piece of hobby board, capturing waveforms using my Rigol DS4022 scope and a Rigol RP1050D high voltage differential probe and the UltraScope software.

For transformers, I used the magnetic transformer from the Elco enclosure, and I bought three ELV’s from eBay, two from a known brand, and one unknown brand:

For bulbs, I bought a variety of models from Amazon, eBay, and 1000bulbs:

  • Sylvania 58327: 50W Halogen MR16, 3000K, 35 Degree, 1450 CBCP.
    These are the halogen bulbs I currently use, about $2.20 per bulb.
  • Torchstar TS010: Dimmable, 12V 4W MR16 LED, 6000K Daylight, 50 Watt Equivalent, 330 Lumen, 60 Degree Beam Angle.
    I ordered a 10-pack from Amazon, the price worked out at about $5.50 per bulb. The packaging is generic, with a black marker dot indicating this to be a “pure white” variant. The bulb itself contains no markings, other than a small Torchstar sticker on the base. The bulb color is very blueish, like that of a daylight compact fluorescent bulb. I found the color to be very displeasing and distracting in my office environment, it made my color calibrated monitor screen appear yellow.
  • Torchstar TS010: Dimmable, 12V 4W MR16 LED, 3200K Warm White, 50 Watt Equivalent, 330 Lumen, 60 Degree Beam Angle
    I ordered a 10-pack from Amazon, the price worked out to about $5.70 per bulb. Like the daylight version, the packaging is generic, with a black marker dot indicating this to be a “warm white” variant. The bulb itself contains no markings, other than a small Torchstar sticker on the base. The bulb color is pleasing, pretty close to the halogen.
  • Soraa Brilliant 00965: Dimmable, 12V 9W MR16 LED, 75 Watt Equivalent, 3000K, CRI 80, CBCP 1540, 590 Lumen
    I ordered the bulbs from 1000bulbs, the price is about $28 per bulb. The color is pleasing but it appears to be ever so slightly bluer, more noticeable when dimmed. This bulb is bright, at 75W equivalent, almost too bright for my office as one of the bulbs is right above my head.
    Soraa specializes in high color quality products, and this model is from the older Brilliant Series, while I was really looking for the new Vivid Series bulbs, like the 00943, but it seems these bulbs are not yet available. I hope to find and test some when they do become available. At the price point of near $30 they are definitely specialty use, but I am interested in the supposed dimmability improvements.
  • Soraa Outdoor 00107: Dimmable, 12V 9.8W MR16 LED, 2700K
    This is a 36W equivalent LED for outdoor use, I bought them for about $24 more than a year ago, the line has since been discontinued.
  • Soraa Premium 2 00249: Dimmable, 12V 11.5W MR16 LED, 3000K
    I bought these more than a year ago for about $34 each,  the line has since been discontinued.
  • Architectural LED MR16-DIM-12V: 2700K 45deg
    I received samples of these MR16 LED’s from my electrician, I could not find any info on them.
  • eBay Dimmable CREE LED COB MR16: 6W MR16
    I bought a batch of 10 warm white and 10 daylight 6W bulbs, and a 9W and a 12W. The 6W bulbs are about $3 per bulb. These bulbs worked surprisingly well and the color was good. Note that the 9W and 12W variants are longer than standard MR16’s.

I tested the transformer response by monitoring the high voltage AC input and low voltage AC output sides using the oscilloscope. I controlled the ELV transformers using the ELV dimmer and the magnetic transformer using the magnetic dimmer. I attached a 12ohm resistor for a purely resistive load, the halogen bulb, and the Torchstar LED bulb. I captured oscilloscope screenshots at full, half, and lowest dimming settings.

Here are the results for the magnetic transformer:

Here are the results for the HATCH RS12-60M-LED ELV transformer:

Here are the results for the HATCH RL12-60A ELV transformer:

Here are the results for the Advance Lite TC60W ELV transformer:

Looking at the results we can see that the response waveforms for the halogen bulb is, not surprisingly, near that of the resistor. We can see that the magnetic transformer and LED load has all sorts of inductive goodness going on. And we can see that the RL12-60W and TC60W ELV transformers are not nearly as well behaved as the RS12-60M-LED ELV that is specifically designed for LED loads.

I then proceeded to test the dimmability of the various LED bulbs, I summarize my subjective findings below:


Magnetic: Good dimming range
RS12-60M-LED: Good dimming range, slight transformer buzzing
RL12-60A: Good dimming range
TC60W: Good dimming range


Magnetic: Good dimming range, flicker at low end
RS12-60M-LED: Limited dimming range, no flicker, slight transformer buzzing
RL12-60A: Good dimming range, continuous flicker
TC60W: Good dimming range, flicker at low end

Soraa Premium 2:

Magnetic: Good dimming range, flicker at low end, very loud transformer buzzing
RS12-60M-LED: Good dimming range, flicker at low end, slight transformer buzzing

Soraa Brilliant:

Magnetic: Good dimming range, flicker at low end, slight transformer buzzing
RS12-60M-LED: Good dimming range, slight transformer buzzing


Magnetic: Good dimming range, switches off before end of dim range
RS12-60M-LED: Good dimming range, slight transformer buzzing

I was surprised that the cheap $3 eBay CREE COB MR16 LED bulbs worked as well as they did. Only downside is they switch off at around 20% when using the magnetic transformer, but dim down well. I don’t know if they really contain CREE COB LED’s, but the COB array arrangement of LED’s provide an even light source.

The Torchstar bulbs have a slight flicker at the low end, but dims down all the way, a bit more expensive compared to the eBay bulbs, but US based Torchstar support may be worth the extra 1$ per bulb.

The RS12-60M-LED ELV transformer performed well with halogen and LED loads, but the buzzing sound with or without load was a disappointment. I tested with two units, both buzz. I contacted the manufacturer to find out if this is normal, or if the units I bought on eBay are faulty.

I have yet to find a MR16 LED that can be driven by a magnetic transformer that performs like halogens, my search continues.

Nest Protect False Alarms

2AM, beep, smoke alarm low battery warning, and when one beeps, all the interconnected ones beep, now it is impossible to find which one has a low battery. As for how smoke alarms look, I’ve always wondered who made those terrible aesthetic design choices, maybe it is some kind of industry insider competition to see who can design the ugliest unit with the most obnoxious markings, and still get them sold.

I was thrilled when Nest announced the Nest Protect combination smoke and CO alarm, finally usability and technology catching up with smoke alarms, and an attractive looking unit. I’ve been a long time fan and user of the Nest thermostats, first one v1 unit, and later two v2 units, and I hoped the Nest Protect would do for smoke alarms what Nest did for thermostats.

I pre-ordered ten alarms from Amazon in October 2013, delivered in December 2013. Installation was easy, but I do wish there was a way to get more spoken locations, e.g. “smoke in kids bedroom”, which kid’s bedroom, wait, let me get my phone to see, not.

A week or two after installation we are having friends over for a barbecue, I show the alarm units, I show the mobile app, I explain how great the wave to silence alarm feature is, and how it will warn you before the alarm sounds, everybody is very impressed. Until a few hours later when one of the units go off, “smoke in the guest bedroom”, what smoke. I wave at it, nothing, I press the button, “this alarm cannot be silenced”. Keep in mind they are all interconnected with a mesh wireless network, so all ten units are screaming. After the kids stopped crying and we moved the party outside, I get a ladder and remove the unit, still screaming, I take it to fresh air, still screaming, I get a screwdriver open it up and remove the batteries, silence, but the rest of the units are still screaming, and pressing the button on those units still say “this alarm cannot be silenced”. About 5 minutes after removing the battery from the failed alarm the the other alarms stop. Egg on my face.

Nest support exchanged the unit and sent out a replacement.

As I was browsing the Nest support forums I noticed many other users reporting false alarms, some reporting that replacement units resolved the problem, some reporting repeat problems. Things got worse for Nest when they issued a recall, offering refunds, disabling the wave feature with a firmware update, and stopped selling units until they swapped stock for units with the newer firmware before re-releasing at a reduced price.

October 2014 early AM the alarm goes off, false alarm again, at least this time the alarm silenced itself after a minute. After some back and forth, and an escalation, Nest support agreed to replace all units. The new units have September 2014 manufacturing dates, so I hope these new units are less buggy.

January 2015 early AM the alarm goes off, false alarm again, this time the alarm stopped after only a few seconds. I’ve had enough, my kids are scared, my wife is mad, Nest, you’re out.

Nest support agreed to issue a refund for all ten units, we’ll see how long it takes to receive the refund. And now I’m in the market for combination smoke and CO alarms again, and there are not many choices, if you want something that is functional and good looking.

I was tempted to wait for the First Alert Wi-Fi enabled combination smoke and CO alarm, available for pre-order on Amazon, and although this unit is from a well established manufacturer, hopefully no false alarms, I’m not making the same mistake I made with Nest. Regardless of the pre-order option, it still leaves me unprotected, and I need something now. I could simply not find a decent looking, combination smoke and CO, interconnectable, and hardwired unit, big problem being decent looking.

In the end I opted for the First Alert PC910V units, they are low profile voice enabled combination smoke and CO units with a built-in 10 year battery, sold at Lowes or Amazon. Not interconnected, not hardwired, but at least they look half decent.

Installing these units turned out to be a bit more tricky than I anticipated. The install base is so small that the round ceiling junction boxes are barely hidden, and the instructions specifically call out that they are not to be installed on junction boxes due to air flow concerns.


Below are some pictures showing the size differences between the Nest base (left, bottom), First Alert base (center, middle), and a round cover plate (right, top):

Base Size Comparison

Base Size Comparison

To account for the junction box ventilation warning I sealed between the junction boxes and the ceiling drywall, and between the cover plate and the ceiling. The alarm bases were mounted on the cover plates, see pics below.

Junction box

Sealed around junction box

Sealed around cover

Base on cover


Due to the small footprint of the alarm, the cover plate and imperfections around the hole in the ceiling can be seen when looking up at an angle. (Sorry for the crappy pictures, iPhone in low night not so great)

Drywall marks

Let’s hope I never hear them peep, at least not for ten years if we can trust the battery life, and at least not without a real emergency.

Rack that server

It’s been a year and a half since we moved into the new house, and I finally have the servers racked in the garage. Looks pretty nice compared to my old setups.

My old setup was as follows:
Two DELL OptiPlex 990 small form factor machines with Windows Server 2008 R2 as Hyper-V servers. One server ran the important 24/7 VM’s, the other was used for testing and test VM’s. The 24/7 VM’s included a W2K8R2 domain controller and a W2K12 file server.
For storage I used a Synology DS2411+ NAS, with 12 x 3TB Hitachi Ultrastar drives, configured in RAID6, and served via iSCSI. The the iSCSI drive was mounted in the Hyper-V host, and configured as a 30TB passthrough disk for the file server VM, that served files over SMB and NFS.
These servers stood on a wooden storage rack in the garage, and at the new house they were temporarily housed under the desk in my office.

One of my primary objectives was to move the server equipment to the garage in an enclosed server rack, with enough space for expansion and away from dust. A garage is not really dust free and does get hot in the summer, not an ideal location for a server rack, but better than finding precious space inside the house. To keep dust to a minimum I epoxy coated the floor and installed foam air filters in the wall and door air inlet vents. To keep things cool, especially after parking two hot cars, I installed an extractor fan. I had planned on connecting it to a thermostat, but opted to use a Panasonic WhisperGreen extractor fan rated for 24/7 operation, and I just leave it on all the time. We have ongoing construction next door, and the biggest source of dust are the gaps around the garage door. I’ve considered applying sticky foam strips next to the garage door edges, but have not done so yet.

In retrospect, preparing the garage concrete surface by hand, and applying the Epoxy Coat kit by myself, is not something I would recommend for a novice. If you can, pay a pro to do it for you, or at least get a friend to help, and rent a diamond floor abrasion machine.

I did half the garage at a time, moving everything to one side, preparing the surface by hand, letting it dry, applying the epoxy and flakes, letting it dry, and then repeating the process for the other side. I decided the 7″ roller that came with the kit was too small, and I bought a 12″ roller, big mistake, as soon as I started rolling the epoxy there was lint everywhere. From the time you start applying the epoxy you have 20 minutes working time, no time to go buy the proper type of lint free roller. I did not make the same mistake twice, and used the kit roller for the second half, no lint. With the experience gained from the first half it was much easier the second time round, and the color flake application was also much more even compared to the first half.

To conserve space in the garage I used a Middle Atlantic WR-24-32 WR Series Roll Out Rotating Rack. The roll out and rotate design allowed me to mount the rack right against the wall and against other equipment, as it does not require rear or side panel access. I also used a low noise MW-4QFT-FC thermostatically controlled integrated extractor fan top to keep things cool, and a WRPFD-24 plexiglass front door to make it look nice.

The entire interior cage rolls out on heavy duty castors, and the bottom assembly rotates on ball bearings. The bottom of the enclosure is open in the center with steel plate tracks for the castors, and must be mounted down on a sturdy and level surface. My garage floor is not level and slopes towards the door, and consequently a fully loaded rack wants to roll out the door, and all the servers keep sliding out of the rails.

I had to level the enclosure by placing spacers under the front section, and then bolting it down on the concrete floor. This leaves the enclosure and the rails inside the enclosure level, but as soon as I pull the rack out on the floor, the chassis slide out and the entire rack wants to roll out the door. I had to build a removable wood platform with spacers to provide a level runway surface in front of the rack, that way I can pull the rack out on a level surface, and store the runway when not in use.

The WR-24-32 is 24U high, and accommodates equipment up to 26″ in length, quite a bit shorter than most standard racks. The interior rack assembly pillar bars are about 23″ apart, with equipment extending past the pillar ends. This turned out to be more of a challenge than the 26″ equipment length constraint. When the rack is in its outside rotated position, the 23″ pillars just clears the enclosure, but the 26″ equipment sticking out past the pillars do not, and prevents the rack from rotating. This requires brute force to lift the castors, and a very heavy loaded rack, over the rail edge and pull the enclosure out all the way before the rack would rotate freely.

Another problem with the 23″ pillar spacing is the minimum adjustable distance for the 4U Supermicro chassis rails is about 25″, and they would not fit between the pillars. I had to order a shorter set of adjustable rails, and use the chassis side of the original rails to match the chassis mounting holes, and the rack side of the rails to clear the pillars, fortunately they fit perfectly into each other, but not on the rack. The WR-24-32 has tapped 10-32 screw holes in all locations, i.e. no square holes anywhere, which meant I had to use my Dremel to cut the quick mount tabs from the rails in order to screw them on instead of hanging them on.

Rather than using another NAS based storage solution I opted for direct attached storage, so I was looking for a 24-bay chassis, less than 26″ in length, with low noise fans. I opted for a Supermicro 4U 24-bay SuperChassis 846BE16-R920B for the main file server, and a 4U 8-bay SuperChassis 745BTQ-R1K28B-SQ for the utility server. It was the SC846’s included rails that were too long to fit between the posts, and I replaced them with a MCP-290-00058-0N short rail set.

I used Supermicro X10SLM+-F Xeon boards with Intel Xeon E3-1270 v3 processors for both systems. Low power and low heat was a higher priority than performance, and the E3 v3 processors were a good balance. I’ve had good experiences with the X9 series SM boards, but I have mixed feelings about the X10 boards. Kingston dropped support for these boards due to memory chip incompatibilities, and SM certified memory for this board is very expensive, and I had endless troubles getting the boards to work with an Adaptec 7805Q controller. The 7805Q controller would simply fail to start, and after being bounced around between SM and Adaptec support, SM eventually provided me with a special BIOS build, that is yet to be publicly updated, that resolved the problem. I had no such problems with the newer 81605ZQ controller I used in the 24-bay chassis.

For the 24-bay system storage, I used 2 x Samsung 840 Pro 512GB SSD drives in RAID1 for booting the OS and for MaxCache, 4 x Samsung 840 EVO 1TB SSD drives in RAID5 to host VM’s, 16 x Hitachi 4TB Coolspin drives plus 2 x hot spares in RAID6 for main storage. The 56TB RAID6 volume is mounted as a passthrough disk to the file server VM. To save power and reduce heat I host all the VM’s on the SSD array, and opted to use the consumer grade Hitachi Coolspin drives over the more expensive but reliable Ultrastar drives. The 8-bay system has a similar configuration, less the large RAID6 data array.

The SM boards are very easy to manage using the integrated IPMI KVM functionality. Other than configuring the BIOS and IPMI IP settings on the first boot, I rarely have to use the rack mounted KVM console. Each server runs W2K12R2 with the Hyper-V role. I am no longer running a domain controller, the complexity outweighed the benefit, especially with the introduction of Microsoft online accounts used in Windows 8. The main VM is a W2K12R2 storage file server VM, with the RAID6 disk in passthrough, serving data over SMB and NFS. My other VM’s include a system running Milestone XProtect IP security camera network video recorder, a MSSQL and MySQL DB VM, a Spiceworks VM, a Splunk VM, a UniFi Controller VM, and several work related VM’s.

I had Verizon switch my internet connection from Coax to Ethernet, and I now run a Ubiquity EdgeRouter Pro. I did run a MiktroTik Routerboard CCR1009-8G-1S-1S+ for a while, and it is a very nice box, but as I also switched out my EnGenius EAP600 access points to Ubiquity UniFi AC units, and I replaced the problematic TRENDNet TPE-1020WS POE+ switches with Ubiquity ToughSwitch TS-8-Pro POE units, I preferred to stick to one brand in the hopes of better interoperability. Be weary of the ToughSwitch units though, seems that under certain conditions mixing 100Mbps and 1Gbps ports have serious performance problems. I am still on the fence about the UniFi AC units, they are really easy to manage via the UniFi controller, but some devices, like my Nest thermostats, are having problems staying connected. Not sure if it is a problem with access points or the Nest’s, as there are many people blaming this problem on a Nest firmware update.

I used an APC Smart-UPS X 1500VA Rack/Tower LCD 120V with Network Card for clean and reliable power, and an ITWatchDogs SuperGoose II Climate Monitor for environmental monitoring and alerting.

After building and configuring everything, I copied all 30TB of data from the DS2411+ to the new server using robocopy with the multithreaded option, took about 5 days to copy. I continued using the old systems for two weeks while I let the new systems settle in, in case anything breaks. I then re-synced the data using robocopy, moved the VM’s over, and pointed clients to the new systems.

VM’s are noticeably more response, presumable due to being backed by SSD. I can now have multiple XBMC systems simultaneously watch movies while I copy data to storage without any playback stuttering, something that used to be an issue on the old iSCSI system.

The best part is really the way the storage cabinet looks 🙂

This is the temporary server home under my office desk:

Finished product:

The “runway” I constructed to create a level surface:

Pulled out all the way, notice the cage is clear, but the equipment won’t clear:

To clear the equipment the castors have to be pulled over the edge:

Rotated view:

The rarely used KVM drawer:

Extractor fans:

Night mode:

Self Signed BitFury Drivers

Almost two years ago I pre-ordered some bitcoin mining hardware from Butterfly Labs, what a waste. After countless delays, more than a year late, they finally shipped the hardware, and given the low probability of ever recovering the money through mining, I immediately sold the hardware on eBay, for a little profit.

In the mean time USB stick miners became available, outperforming GPU mining, and easy to setup and run. I’ve had a couple of ASICMiner Block Erupter’s running under my desk for some time, in the early days I saw some fractions of coins coming in, but in recent months they are so under-powered against the current hash-rates that they do little more than blink lights.

There is a resurgence in USB stick mining hardware, specifically the Bitfury type devices, many based on the NanoFury open source project that provided software, design, and PCB schematics.

I got myself a Red Fury, an Nano Fury II, and a Hex Fury. Compared to the 300MH/s of my little Block Erupters, these run at 2GH/s, 4GH/s, and 11GH/s respectively. There is still no way to ever make a profit in mining (at this scale), but I was really interested in seeing how these newer generation devices worked, especially since the publication of the NanoFury open source project, where in theory I could build my own.

So what does this have to do with self signed drivers, well, my mining tool of choice is CGMiner, but CGMiner currently only runs Nano Fury II’s at half speed, requiring the use of BFGMiner to go full speed. But unlike CGMiner that accesses all USB devices via Zadig installed WinUSB drivers, BFGMiner requires native Windows drivers, and neither the Red Fury nor the Hex Fury drivers are signed, so no installation on Windows 8 x64 (without disabling driver signing on every boot).

Looking at the INF files, all these devices do is register the USB hardware id as a generic null modem USB to COM bridge device, so no binaries required, just a signed CAT file.

After a bit of searching I found that I was not alone in my frustration, and I found a self-signed Red Fury driver. But, the Hex Fury used a different hardware id, and most people used CGMiner, so no need for a signed native driver as Zadig took care of that for us. So, I created my own signing script and signed my own drivers, install ok, BFGMiner happy.

If you just want signed drivers, get a copy of self-signed “Bitfury BF1” and “bi•fury” drivers here.

If you are interested in signing your own drivers, read on.

I tested on Windows 8.1 Update 1 x64:

Install the Windows 8.1 SDK and WDK.
Get the original “Bitfury BF1” and “bi•fury” INF files.
The bf1.inf file is saved in *NIX format (CR), convert it to Windows format (CRLF).
Create a self signed certificate:

makecert.exe -r -pe -ss PrivateCertStore -sr localMachine -n "CN=BitFury Test Signing Certificate" "C:\BitFury\BitFuryTest.cer"

Self Signed Certificate

Prepare the INF files, and create CAT files:

stampinf.exe -n -f "C:\BitFury\bf1.inf" -d * -v * -c ""
stampinf.exe -n -f "C:\BitFury\bifury_c4C.inf" -d * -v * -c ""
inf2cat.exe /v /driver:C:\BitFury\ /os:7_x86,7_x64,8_x86,8_x64

Sign the CAT files:

signtool.exe sign /v /s PrivateCertStore /n "BitFury Test Signing Certificate" /t "C:\BitFury\"
signtool.exe sign /v /s PrivateCertStore /n "BitFury Test Signing Certificate" /t "C:\BitFury\"

To use the drivers, you have to import the signing certificate into the local certificate store. As this is basically a self-signed CAT file, there are no trusted root certificates in the system that signed the signing certificate, and we need to add the signing certificate to the root and the trusted certificate stores.

certmgr.exe -add -c "C:\BitFury\BitFuryTest.cer" -s -r localMachine root
certmgr.exe -add -c "C:\BitFury\BitFuryTest.cer" -s -r localMachine trustedpublisher

Alternatively you can run “certlm.msc”, and import the certificate file into the “Trusted Root Certification Authorities” and the “Trusted Publishers” hives.

SelfSign.2 SelfSign.1

Last thing left to do is to use your newly signed drivers when selecting the custom driver from device manager.


Here is a package with signed drivers, and scripts to help you sign your own INF files, and import the certificate. It should work on Windows 7 and Windows 8 x86 and x64. Your mileage may vary, use at your own risk 🙂

Electrical Power Quality

Earlier this year we moved a couple miles from Redondo Beach to Manhattan Beach, bigger house, better school district.
As far as the house and area is concerned, it is definitely an upgrade, but not so for the utilities.

Monthly utilities are a lot more expensive, not so much the per unit fees, but the base service fees, not just a couple $, but three of four times what we paid in Redondo Beach. Now, if it came with better offerings, or better service, or higher quality, ok, but the opposite.
Water quality is worse, specifically hardness, MB supplies its own water, RB gets water from LADWP, and that unsightly water tower that no longer serves any practical purpose, with efforts to demolish it always being thwarted.
As a new resident trash collection makes me pay almost thirty $ extra per month for an extra trash can, while grandfathered-in residents keep extras for free. Now, I know it is unfair to judge a service by their employee’s actions, or is it, but the trash collection guy is a jerk, if a little dust and having to get out of the truck is going to get you agitated, you are in the wrong business, especially when compared with the pack of trash collection men in RB that were always friendly and willing to give a hand.
But, I really digress, I want to discuss electrical power quality problems.

In the six plus years we lived in RB, I think we had one scheduled power outage, and maybe two short unplanned outages. Since moving to MB earlier this year, we’ve had two scheduled outages, one lasting an entire day, and several unscheduled outages.
The power is unreliable, SCE knows it, the city knows it, there are some plans addressing it, see here here here here.

My concern is not really power being on or off, it is power being on but of poor quality; an electronic equipment killer.

When we moved in, the first signs of electrical problems were flickering lights. At first I thought it was a problem with the Vantage light control system, but even lights directly on utility power flickered. As soon as I hooked up UPS’s to my servers and the signal distribution system, the UPS’s started complaining about power quality. Occasionally during the day I would get a notification from the UPS’s that it detected a distorted input, and every night the UPS’s would complain about low input voltage.
It may be coincidental, but I’ve also had two astronomical clock light timers fail at the same time, the casings were scorched in what appears to be signs of electrical damage.

UPS Event Log:
APC Event Log

In order to quantify the problem, I used a Fluke VR1710 Voltage Quality Recorder. The device plugs into a mains outlet, and records events, and a USB port is used to configure the device, and download recorded data.

As I am not a power quality expert, I referred to Wikipedia to and Power Quality In Electrical Systems for information and reference material. To further simplify the analysis, I opted to compare my office power with my home power, this allowed me to easily visualize the quality differences, granted, I am assuming my office power is good.

I configured the VR1710 to take measurements every 10s, and to record exceptional events, about 10 days worth of data. I set the dip threshold to 106V, the swell threshold to 127V, and the transient sensitivity to 5V.

VR1710 Settings:
VR1710 Settings

Below are reports detailing the recorded events, click graphs to view full resolution:

Home Voltage:
Home - Voltage

There is a clear pattern of voltage drops below 102V every evening, these drops are also observed in the UPS logs showing low voltage warnings around 7:30PM every evening.

Office Voltage:
Office - Voltage

The office voltage is very stable.

Home Flicker:
Home - Flicker

According to Wikipedia and PQW short term flicker (Pst) is noticeable at values exceeding 1.0, and long term flicker (Plt) is noticeable at values exceeding 0.65. These results would explain why we observe lights flickering.

Office Flicker:
Office - Flicker

Office flicker values are well within acceptable ranges.

Home Statistics:
Home - Statistics

From this distribution we can see the wide spread in voltages, well below the 120V theoretical norm. This chart does not show it, but the 95% distribution is 115.5V, and the 5% distribution is 106.1V.

Office Statistics:
Office - Statistics

The office voltage distribution is nicely clustered around 119V, with the 95% distribution at 119.6V, and the 5% distribution at 117.4V.

Home Dips And Swells:
Home - Dips Swells

ITIC and CBEMA are standards for acceptable power quality, see here for a detailed description.
To describe the graph, I quote from the Fluke Power Log software manual:
Dips and swells are shown on a CBEMA (Computer Business Equipment Manufacturers Association) and ITIC (Information Technology Industry Council) plot classification table according to EN50160. On the CBEMA (blue) and ITIC (red), curve markers are plotted for each dip and swell. The height on the vertical axis shows the severity of the dip or swell relative to the nominal voltage. The horizontal position shows the duration of the dip or swell. These curves show an ac input voltage envelope which typically can be tolerated (no interruption in function) by most Information Technology Equipment (ITE).

Based on the graph we can see a large number of events exceeding the acceptable ranges. Since there were no dips at the office, there is no graph for the office.

Home Transients:
Home - Transients

I only show the transients graph for home, as the wave forms all look different, and the only difference between home and office is 87 events were recorded at home while 10 events were recorded at the office for the same approximate time duration. See PQW for an explanation of transients.

We can clearly see that the power quality at my house is significantly worse compared to the power at my office.

I am speculating, but I wonder if the old transformer across the road can supply sufficient power, given that it used to supply power to three small very old houses on four lots, demolished to make room for four new larger houses?

I just opened a support ticket with SCE, let’s hope they can do something about the problem.