Yesterday I enjoyed the webcast by Hariharan Sethuraman and Chris Haslam, both from Microsoft. They talked about ICE, which stands for Information Security Consolidated Event Management System. ICE collects the in- and outbound e-mail traffic, login events and web browsing (web proxy and firewall logs) and stores it for 60 days, to provide an audit-trail in case of security events. The webcast was about how ICE 3.0 was designed and built on top of the infrastructure below.
Imagine the numbers; 40 TB designed -- 27 TB allocated (FibreChannel SAN-)storage, designed to load 60 GB/hour into staging tables -- currently receiving max 1.2 TB daily with 600 GB as daily average, table partitioning, 4 (multi-core?) x64 processors & 32 GB RAM for the Database Engine + 4 (multi-core?) x64 processors & 8 GB RAM for the Integration Services. Accessing all that data via Ad-hoc queries and Report Server reports.
Already ICE version 4.0 is envisioned, however again only for internal usage, ICE is not (yet?) planned as a commercial product. So with ICE not being planned as a product, I just had to ask "Sounds like a great reference project for SQL Server 2005 and is a very useful application, are you planning a white-paper?"... Turned out the webcast is a precursor to the white-paper!!!
I'm looking forward to update this post with the link to that white-paper. For now I can only point you to the on-demand webcast: How Microsoft IT Uses SQL Server 2005 to Power a Global Forensic Data Security Tool (Level 300) (~60 minutes)
/* UPDATE 2007-11-22 */ Link to the afore mentioned white-paper.