My torn data pages - Hardware

After replacing a Hyper-V (windows Server 2012) host, DHCP started to have issues. Unpredictable results so to say.

The old host had a teamed dual-port Intel PT1000. At the network side, a Link Aggregation Group was defined on the Cisco SG300 switch.The new host is equipped with an on-board Broadcom NetXtreme controller and an additional dual-port NetXtreme II adapter. The onboard controller was configured as IP-interface for the host, for the dual-port adapter I installed the Broadcom Advanced Control Suite 4 (BACS4, Version 15.6.31.0) and an additional LAG was defined on the SG300. Well, it wasn’t a clear victory! When pinging between hosts, packets between physical and virtual hosts seemed to be dropped, DHCP wasn’t renewing and RDP-sessions to virtual systems were having response issues (can happen when packets are dropped). The main difference to me was the Hyper-V host, where in the old situation the Intel PT1000 was shared between the host OS and the guests (as external virtual switch). In the new situation the dual port Broadcom was dedicated for the virtual switch.I must admit knowing little more than the basics on Hyper-V thus one Googles in all directions. And thus I stumbled upon SR-IOV. It’s late, SR-IOV made sense to my new situation thus I gave it a try by replacing the old virtual switch with a new SR-IOV based virtual switch. Hurrah, everything seemed to work now; ping, stable RDP sessions and DHCP renewals.

However after a couple of days, I wanted to use my Android tablet, but it did not receive an IP-address. Pressed for time, I switched to my laptop. A few days later, I turned on a Windows 8 system that had been unused for the last couple of weeks and it too did not get an IP-address… so I still did have a DHCP issue! I blamed the switch configuration for a while, as well as the firewalls on the Windows systems. Wireshark showed the DHCP Discover leave at Windows 8 hosts and never arrive at the virtual DHCP server. But other DHCP packets (renewals) were observed, both at the unwilling client and at the server. When I put a static IP on the Windows 8 host, it still couldn’t ping the DHCP Server (or any other virtual system). The MAC of the Windows 8 host did however show up in the MAC table of the switch (but so did the MAC addresses of the virtual servers). I must have reconfigured the switch 13 times, disabled the firewalls and still no result. Oh my… do I now have one of those nasty interoperability issues between switches of different vendors (Cisco and Microsoft)… I needed to learn more about virtual switches in Hyper-V and luckily one of the first links I hit was Hyper-V and VLAN’s by Aidan Finn. I especially want to quote him on:

“I must warn anyone reading this that I’ve worked with a Cisco CCIE while working on Hyper-V and previously with another senior Cisco guy while working on VMware ESX and neither of them could really get their heads around this stuff.  Is it too complicated for them?  Hardly.  I think the problem was that it was too simple!  Seriously!”

That was comforting, it must therefore be that the solution should be simple. At the bottom of Aidan’s post was yet another link that seemed useful Windows Server 2012 NIC Teaming Part 6 – NIC Teaming In The Virtual Machine, and it was! The final part of a series useful articles, which led me to the Microsoft un-support statement of third party NIC teaming: Microsoft Support Policy for NIC Teaming with Hyper-V

Bottom line: the Broadcom Advanced Control Suite could be the source of my problem. I uninstalled it from the Hyper-V host and then configured NIC Teaming from Hyper-V. And the result of ipconfig /renew on the Windows 8 computer: Discover, Offer, Request, ACK. Just the way it should be.

I've been holding this one off for quite some time, but finally bit the bullet... migrate the server.Coming from Windows Small Business Server 2003 (32-bit) going to Windows Server 2008 with Exchange 2007 (both x64), including transferring the web site. There were a few nasty pieces in the process and I didn't have/take the time/resources to test it all though before jumping in... so I got a disruption of about one hour for the blog and inbound smtp.

First issue I encountered was the installation of the server, which I ordered from Dell without operating system. I am a Microsoft Action Pack subscriber, so I slammed the 64-bit Windows Server from the Action Pack on the system, entered the Product Key and then went on to phone based activation. After putting in the last confirmation code and pushing the next button... my activation wasn't accepted!!! ... to cut a long story short, after dozens of voice menu selections and 4 persons, I found the guy who could help me out: Philip. Though it still toke Philip and me about 20 minutes to find the actual problem... I wasn't an Action Pack subscriber anymore!!! Technical story here, but it happened when Action Pack was integrated more tightly with the Partner Program. Somehow I didn't complete the Action Pack renewal transaction completely in February, but the partner site didn't show that (it only showed I'm still good for the Partner Program).

Couple of days later, I could go ahead with the migration process. It turned out to be easier than I expected after I found a very useful document about Upgrading Small Business Server 2003 to Exchange 2007. Some pointers on using the document though;

• Transferring the the other FSMO's (Schema Master is described); Determining FSMO Role Holders
• With Windows Server 2008 you NEED the Exchange 2007 SP1 DVD, Exchange 2007 without SP1 will not pass installation checks.
• Section Migrate mail send does not cover migrate mail receive. This caused me a bit of downtime... The issue was
  SMTP error from remote mail server after MAIL FROM:<abcd@efg.hi> SIZE=2703:
  host abcdef.ghi [10.10.10.10]: 530 5.7.1 Client was not authenticated
  And can be solved in the following manner in Exchange Management Console:
  • Go to the Server Configuration, Hub Transport. Select your server and add Anonymous to the Default SERVERNAME Receive Connector. While you're there, you can increase receiving message size here too.

And there were parts I did different;

• I continue to rely on Vamsoft's ORF for the anti-spam. Version 4.3 is fully compatible with Windows Server 2008 and Exchange 2007 and an ease to configure.
• Since I will continue the same DNS name for OWA and Windows mobile Active Sync, I transferred the Self-signed certificate from the SBS2003 (find it in your IIS, web site, directory security tab) to the new server.

My other bit of downtime was the blog... simply copy from the old server to the new one didn't cut it. So I downloaded the latest release of dasBlog from Codeplex, copied it into the virtual director of new server. Next I had to;

• Copy my own theme
• Compare (and adjust) the \SiteConfig files
• Transfer \Content files and \Logs files
• Make sure the account running the Application Pool has read on all dasBlog folders and change on \SiteConfig, \Content and \Logs.
• The Application Pool is running in Classic Managed Pipeline Mode

Guess I can say I didn't plan for the blog to be upgraded... that just had to happen.

Hmm, did this about half a year ago and didn't blog about it... stupid, because now I had to figure it out again. The unfortunate event was the phone on my PDA (T-Mobile MDA Vario II) had a bit of a malfunction. It either didn't connect to the GSM-network or when it did, calls would be disconnected after a couple of minutes. So I called T-Mobile and after having determined the problem was the phone (not the SIM), they collected it for repair. Naturally, I made a backup of my personal stuff and then removed the personal stuff and security settings from the device before it was picked up (hey, I do security training occasionally).

Well today I got my device back... software update, all information gone (but I have a backup)... I just had to restore my stuff and configure network settings and Exchange synchronization again. No problem until: 0x80072FD; indicating the certificate on the Exchange Server (my Small Business Server) was invalid.

The solution is to add my SBS self-signed certificate to the trusted root certificates of the PDA. How?

Well first make sure you environment is set up for synchronization. Not my issue, but it never hurts to check on it. See Petri for the overview.

Now obtain your SBS self-signed certificate, in all cases this will be located on the virtual directory Exchange on your web server. From IIS Manager you can directly export the certificate (without private key) to the required DER encoding. Execute following steps, which are derived from kb841060:

1. Export the root certificate to a computer that is running Microsoft Windows in DER encoded binary X.509 format with a .cer file name extension.
2. Create a root-folder Storage on the Windows Mobile device.
3. Download and extract SmartPhoneAddCert.exe package on your Windows Computer.
4. Use ActiveSync (Windows XP and earlier), Windows Mobile Device Center (Windows Vista) or the storage card to transfer the DER-encoded certificate and the SPAddCert.exe (from the downloaded package) to the \Storage folder on your Windows Mobile device.
5. On your Windows Mobile device, start SPAddCert.exe from the \Storage folder. It will give a warning that the application is not verified (or something the likes), ignore the warning and proceed. The application will now show you the certificate you exported, continue by confirming all actions.
6. Restart your device.

You can check the trusted root certificates under Settings, System, Certificates, Basic (in my case (Dutch) Instellingen, Systeem, Certificaten, Basis). After executing above procedure you should see the SBS-cert here.

I'm not sure if Windows Mobile 5 really required the restart, but that's because before I restarted I received the error 0x85010014 from ActiveSync on my SmartPhone. After restart (still the same error) it turned out to be a connection issue. Due to firewall restrictions I had to use another interface on my multi-homed SBS Server (hey, it's been a couple of months!!).

Next error I encountered from ActiveSync was 0x85020013, but since I had that funny feeling I mistyped my password (big fingers, small keys), that one was quickly resolved.

Device details:

Windows Mobile 5.1.195 (Dutch)

T-Mobile MDA Vario II (device modelnr: HERM300)

Yesterday I enjoyed the webcast by Hariharan Sethuraman and Chris Haslam, both from Microsoft. They talked about ICE, which stands for Information Security Consolidated Event Management System. ICE collects the in- and outbound e-mail traffic, login events and web browsing (web proxy and firewall logs) and stores it for 60 days, to provide an audit-trail in case of security events. The webcast was about how ICE 3.0 was designed and built on top of the infrastructure below.

Imagine the numbers; 40 TB designed -- 27 TB allocated (FibreChannel SAN-)storage, designed to load 60 GB/hour into staging tables -- currently receiving max 1.2 TB daily with 600 GB as daily average, table partitioning, 4 (multi-core?) x64 processors & 32 GB RAM for the Database Engine + 4 (multi-core?) x64 processors & 8 GB RAM for the Integration Services. Accessing all that data via Ad-hoc queries and Report Server reports.

Already ICE version 4.0 is envisioned, however again only for internal usage, ICE is not (yet?) planned as a commercial product. So with ICE not being planned as a product, I just had to ask "Sounds like a great reference project for SQL Server 2005 and is a very useful application, are you planning a white-paper?"... Turned out the webcast is a precursor to the white-paper!!!

I'm looking forward to update this post with the link to that white-paper. For now I can only point you to the on-demand webcast: How Microsoft IT Uses SQL Server 2005 to Power a Global Forensic Data Security Tool (Level 300) (~60 minutes)

/* UPDATE 2007-11-22 */ Link to the afore mentioned white-paper.

 

This morning, as I wanted to start working, I noticed my server wasn't working... Outlook couldn't connect to Exchange and the ReSQueL website was down as well. The server still had power, but no response (not even the keyboard LED for NumLock). So a power-cycle was next, well at least half of it, power down worked, power up didn't.

Absolutely nothing happened! Now I must admit, I've been there before with that type of system (not just my own). So I pulled the power cord and left the system for about 10 minutes, then tried again. This time I heard some ticks coming out of the power supply unit. And believe me, ticks from the PSU is not a good sign about the health of the thing.

However, I did have another unused ATX cabinet with PSU, so I tried transferring the internals of the old system to the replacement cabinet. Unfortunately, the TP123 motherboard has an extra power connector for the CPU core voltage. So putting power on the board worked, but (as expected) it didn't boot.

Not wanting to extend the downtime much further, I grabbed the car-keys, drove to Office Centre (cash and carry beats a webshop when you need something NOW!!!) and bought the cheapest config with 1 GB internal memory. Back home, I tested if the system worked. It did. Next I added the old disk and an extra network card and booted the system from the old disk... BOSDed and rebooted right away. But since the config is totally different, that shouldn't be too surprising. So I fetched the Windows Small Business Server 2003 R2 DVD and started the repair installation. Finally some time to sit down and write some... if all is well, this is on the blog (and the blog available) little over 5 hours 6 hours after I noticed the server down... always fun when you don't have the drivers for your new system (only the Vista drivers supplied)... NOT!

Sometimes there are those things that annoy you, like the amount of memory seen by my computer. I have a HP Compaq nx6325 equipped with 4 GB of RAM, but Vista only reports 2943 MB. Not that I expected to see 4096 MB, I'm a bit smarter than that. This machine has an ATI Radeon Xpress 1150 which has no memory of its own and uses HyperTransport HyperMemory to share the system memory between CPU and GPU. But why should the GPU chew up over 25% of the systems memory?!?! (And why would I want to limit the amount of memory used for the GPU? I'm running Vista and SQL Server Developer x64 Editions, and would like to have maximum memory for the database services.)

Well, today I read a paper from Microsoft explaining what is going on with memory. Not that I have a solution now, but at least I can ask smarter questions now.