Biz Intel Expanded in Microsoft's SQL Server

Biz Intel Expanded in Microsoft's SQL Server: "Kirsten Ward, lead product manager for SQL Server at Microsoft, said the report packs, or templates for commonly used reports, come as a result of the overwhelming success SQL Server 2000 Reporting Services has had since it launched last January -- more than 100,000 downloads.
One pack is tailored for Microsoft Exchange Server; the other for Microsoft CRM (Customer Relationship Manager application). The pack for Exchange features 13 templates, or typical reports that Ward said people managing Exchange implementations want to have.
Ward said an example would be a template that allows users to scan the e-mail activity across a company and see who has the largest inboxes. The CRM pack comes with six templates, including account details and lead summaries. "

MDX in Analysis Services: Mastering Time: Moving Averages - Another Approach

Get your feet wet in MDX. Maving averages are one of the cool ways to demonstrate the power of MDX in Analysis Services. So head on over to: MDX in Analysis Services: Mastering Time: Moving Averages - Another Approach.

Rittman: Colin White : "Now is the Right Time for Real-Time BI"

Mark Rittman points out a DMReview article, Colin White : "Now is the Right Time for Real-Time BI".

I have a little first hand experience creating custom BAM solutions, and when they work right, the results are transformative in nature. After all, much of a line manager's job involves reacting to problems, or in figuring out what the overall picture based on very subjective input they get from from employees or customers. BAM can take all the guess work out of an assessment of the state of a process, and it can even be designed to automatically take action.

Take a large-company IT Help Desk for example. A BAM solution can keep track of ticket levels, and do it by application, and compare it to a daily average for that application. If a certain app is producing more tickets than normal all of a sudden, this might be missed completely within the Help Desk application, which is designed to improve the productivity of Call Reps (which most do quite well), but not spot such high level things. With BAM, the problem could be identified immediately, alert the Help Desk Manager with an email or page, and even fire off an alert to group that manages that particular application (and maybe notify customers proactively, before they start hitting the phones to complain). The best solutions allow this to be done without any changes to the underlying application or workflow, so like many good BI projects, it essentially leverages the existing data stored in the application to generate processed, actionalbe intelligence.

That's what I've seen a BAM solution do... a good one can change a stale, reactive enterprise application that handles workflow well into a living, breathing, active application that manages the process itself, plus everyone that works in it, at least much more so than before. It frees up managers to concentrate on more important things than trying to figure out by looking around the room if there is a problem right now.

Or maybe, can eliminate them? It's possible. It definitely wil change their focus.

Reading Tea Leaves, or Business Intelligence ROI

I have been doing a little research on different approaches to evaluating the ROI of BI projects. My company has a new product we are getting ready to start selling, and we are sure it's a winner. Just about everything is in place technology wise, and it's a compelling pitch that with obvious value, I think. One thing I'd like in place before we start pushing it is some sort of way to show projected savings, payback, and ROI for a given customer, to make the business case all the more painless. But boy, its frustrating as hell.

There has been plenty written on this subject, which all concludes the same thing - there's a whole lot of guesswork involved, and alot of frustration from business leaders that promised savings never get delivered. As this CFO.com article says: "Five years from now they're sitting there with all our money in their pockets and I'm still trying to figure out how to measure something like 'employee empowerment.'" How do you quantify faster decision making? Better decision making from more transparency? It would be nice if it was as straightforward as this example from Baseline.

My inspiration for finding good, solid numbers to go with is a company we've done some consulting work for, MedMined. Their site and their brochures are chock full of real costs, percentages, and measures that can be evaluated by any interested party. I'd like to come up with similar clarity, but it's tough to get past all the "soft" benefits. The best help I've found is this spreadsheet from Nucleus Research that walks you through all the costs and benefits. Sure it requires you to pull some numbers out of thin air, but it makes it look real thought out.

One thing is certain though, and that is that none of this is certain. If the market for "improving your roi" on existing investments is any measure, then no one is close to figuring this out yet.

Oracle's Move Into Content Management Software? and other CMS stuff

Mark Rittman had this posted a few days ago: Project Tsunami : Oracle's Move Into Content Management Software? It's a big game, and nearly every company (at least at the grass roots techie level) realizes they need a Content Management and filesharing system.

On the Microsoft side, here is a really complete Sharepoint presentation, SharePoint Implementations in the Real World.

When considering such a system, by far the best place to start is with Windows Sharepoint Services. It is free and very well integrated into Server 2003. If Windows 2000 is what you have, Sharepoint Team Services comes free with Frontpage 2000 and up. Both receive rave reviews, and they require no training and no technical knowledge to use. It's as intuitive as you can get.

And before looking at a larger solution (costing 6 figures, all of them will), be sure to consider the open source alternatives. For a very long time Zope was considered the most advanced and extensible CMS available anywhere, and also look at php-Nuke, TikiWiki, etc, etc, etc. There's a list on the left column here. Because these existed in the open source community long before the big vendors caught on, and Free is not a dirty word.

AOL launches program for AIM bots

InfoWorld: AOL launches program for AIM bots: : "Robot operators agree not to distribute users' screen names and pledge to only use the robots for their stated purpose, so that AIM users don't face the type of spam nuisance affecting e-mail, Curry said.
'That's a point of great sensitivity for us,' he said, adding that AOL monitors closely the use of robots to ensure that the experience remains a positive one for users."

Good call. I've been talking to AOL about trying to get the SDBA framework officially authorized, but the open source nature of it is a tough hurdle to cross. It just may work out though.

MSN is a bit behind the curve on this one. Last time I spoke to them, they were solely focused on IM bots for the Enterprise, with no interest in the consumer space.

The Google Cluster Architecture

Via Sadagopan's weblog: Here's an article that's been floating around for a year written by three of Google's architects, written for IEEE. Alot of detail on why it works like it does, and how they can scale up to tens of thousands of servers without exponential costs in management tools.

Grid is huge hype right now, and no one is delivering it right into businesses. But it is coming... and one of the primary applications for it wil be databases as the explosion of RFID and transactional data continues to build up, and BI moves more and more forward to unlocking valuable information out of it. When it does become a topic in IT departments, you'll be glad that you read this pdf and understand the Google model inside and out. Surely, anyone that wishes to be considered knowledgeable in the area of Grid computing will be expected to know how things are done there.

How will "Grid computing" look when it is ready for prime time in the world's largest companies? Only a couple of things are for sure - when it comes there will be a huge demand for skills and knowledge, and solutions won't look anything like they do right now. My personal belief is that the Google path - open source components and commodity hardware - will win out over propietary solutions that are burdened with millions in hardware and software requirements.

Consider:

The cost advantages of using inexpensive,
PC-based clusters over high-end multiprocessor
servers can be quite substantial, at
least for a highly parallelizable application like
ours. The example $278,000 rack [from RackSpace.com] contains
176 2-GHz Xeon CPUs, 176 Gbytes of
RAM, and 7 Tbytes of disk space. In comparison,
a typical x86-based server contains
eight 2-GHz Xeon CPUs, 64 Gbytes of RAM,
and 8 Tbytes of disk space; it costs about
$758,000.2 In other words, the multiprocessor
server is about three times more expensive
but has 22 times fewer CPUs, three times less
RAM, and slightly more disk space. Much of
the cost difference derives from the much
higher interconnect bandwidth and reliability
of a high-end server, but again, Google’s
highly redundant architecture does not rely
on either of these attributes.

When one of the world's fastest, most reliable, and heavily used sites on the internet does not use a SAN (or even SCSI drives), doesn't pay for for redundant power supplies, and pretty much uses all the same parts you and I could buy on Pricewatch, it's something that will be felt across the high availability industry in the future. And our industry is probably near the top of the list where changes will be seen first.

Intelligent Enterprise Magazine: The Land Beyond Transactions

Stewart McKie writes in Intelligent Enterprise Magazine: The Land Beyond Transactions about two kinds of cutting edge analytics, both have been covered on my blog a bit. He notes that with the exploding volume of data of RFID, "Location Analytics" will play a factor, as many metrics for measuring supply chain performance will be available. Being able to visualize metrics by location would no doubt lead to insight, a lesson which can be applied to almost any collection of data with location information.

He also writes about "Metadata Analytics", where previously unstructred data is now available with XML schemas. I agree this is a totally untapped source of info right now. The two examples he gives are XBRL (which I've written about before) and RSS/weblogs. This is actually something I've been thinking about too lately (deja vu), as some services like Feedster let you quickly dig through thousands of logs for mentions of, say, companies of industries. The specific application of it I was considering was a way to integrate such feeds into an SFA or CRM application, so, for example, a sales rep could pull up the summary info for their customer, and also see the latest blog news mentions and rumors about that customer (which I assume to be a company). Stewart mentions a few other cool possibilities... including innovation tracking and "reputational management." Lots of untapped potential there, that's for sure.

Building Scalable Solutions with Windows Datacenter

While much of the rest of the world is gravitating toward grid technologies of one kind or another, Microsoft's bag is still to scale vertically when it comes to the database. With hardware prices on a perpetual downslide, at some point expertise in both areas will be very necessary.

Building Scalable Solutions with Windows Datacenter is a webinar that's easy to follow that has an overview of the version of Windows 2003 Server designed for up tp 64 CPUs, and 512GB of RAM. It's still very rare, but will probably become more common place as SQL Server continues to nibble market share from other vendors in the high-end space. Better to get a few key points on what makes it different now before your boss asks you your opinion.

One claim in the webinar is that just upgrading to Windows 2003 increases disk throughput by 100%. Sounds a bit crazy to me, but benchmarks are quoted. And something to note that I didn't see mentioned is that performance is very closely tied to hardware, and therefore Microsoft does not support customer installations of Datacenter. In this sense it is sometimes referred to as an "Intel Mainframe" operating system. That means you can only get such a machine preloaded from the vendor (which are very few at the moment, including Unisys). The only company I personally know of using Datacenter is EBSCO, the largest magazine publisher in the US. I'm sure there are plenty of others though.

Data Warehousing in Universities

I was doing some research for a demo for a local University, and ran across this study. It has a ton of information on adoption rates of Data Warehouse efforts broken down by institution tyoe and size.

For example, despite that 75% of institutions either have one or are planning one, only 10% report their data warehouse includes data from more than 3 applications. Most popular dimensions? Student, followed by Financial, followed by Human Resources, with Aid, Facilities, and Grants coming next.

Alot of insight here.

Next couple of days might be exciting where I live (in Birmingham, Alabama). Hurricane Ivan is planning to wipe out the whole state. The Governor has closed an interstate, making all the lanes go North to speed up evacuations, which I find pretty cool.

The Business Value of the Microsoft Platform presentation

For your dose of Fear, Uncertainty, and Doubt for the day, here's a partner presentation from Microsoft: The Business Value of the Microsoft Platform. It promises facts to set the record straight about Linux. To sum up the presentation: Linux costs more and MS is better in every way.

MSNBC - Oil Isn't Going Away

I spotted this via Sadagopan's weblog, MSNBC - Oil Isn't Going Away is an interview with the CEO of ExxonMobil. He talks covers a whole range of topics quickly, including the reality vs the hype of alternative energy, the viability of "energy independence" (it's not) and the growth in oil usage. Within a few decades, the world will be using over 330 million barrels/day, equivalent to 7 or 8 times Saudi Arabia's output today. That's alot of oil. He's pretty confident that it's there, and recoverable.

The most important thing isn't the volume of oil, but the infrastructure to transport it and use it, and the systems to manage all of it. It's an incredible problem, and one I think the interview does a good job of framing.

Interesting article, and a reminder that nearly every major industry is going to increase exponentially over time. The relevance? Yhey all will depend on the Data Warehouse more and more.

MDX Essentials: Numeric Functions: Introduction to the AVG() Function

This is an article in Database Journal: MDX Essentials: Numeric Functions: Introduction to the AVG() Function. MDX is is the secret sauce to working magic from OLAP in general (and Analysis Services in particular), so if you have not looked into it yet, chek out the article to get your feet wet.

Sybase Releases Free Enterprise Database on Linux

Slashdot reports that Sybase Releases Free Enterprise Database on Linux. Not open source, but as a free product. Sybase is not that uncommon as a data warehousing platform in very large businesses, and the skills are rare (only 10,000 sybase certified DBAs, vs over 120,000 MCDBAs), and Sybase DBAs get paid quite well, so this could be a way to sharpen up some skills with a certification without having to shell out the investment for the software.

Slashdot suggests that this could be targeted at SQL Server (since they are very similar - SQL 6.5 was purchased from Sybase, and both use the same TSQL syntax for stared procedures), but I doubt that. Seems to me more oriented at the Open Source options that run on Linux (and are eating up that market share).

Innovative IM bots

There are plenty of IM bots out on the internet. Here's a list of AIM bots, and of MSN bots (click on "Intelligent chat bots" on the left). Most are pretty straight forward, with small games and simple Eliza-style jabber. Although I personally don't like bots that try to emulate a real person very much (I don't think it works and they end up annoying, and they just plain aren't innovative), apparently alot of other people do. Here's the funniest one of those I've seen.

Every now and then I hear of a truly useful and cool one. I'm the author of a project that makes it easy to build functional bots, so from time to time I'll hear of someone using it in ways I never considered. What follows are some that I've heard of (some of the names are not included as they are meant to be stay private).

In line with the main theme of this blog, oraclelarrybot (on AIM), constructed by Chritopher Whitely, gives explanations and information about Oracle error codes. In his words:

His account name is oraclelarrybot and can accept the this commands: /help and /error. Here is an error example. /ERROR ORA-00600

Then, I've been told of some neat uses for public facing bots.. such as in Sweden a hacked version of the IM Broadcast app is used to send out announcements of spontaneous Raves. Several games have been put up, including a clever version of hangman in portuguese on MSN (wish I could remember the address!). Just yesterday I learned that a Dutch site, Roadside.info, is using a bot to deliver the latest list of speedtraps in Holland over MSN (they were already using SMS). It's a very busy bot. Also, the RSS feed-to-IM app piece is floating around in a number of places.

The Enterprise space is where nearly all the VC money is right now for bots. Companies like Conversagent have settled into a comfortable space, making bots for help desks and HR information. On the cutting-edge, I already mentioned SPSS is experimenting using IM in their MSurveys suite of products. I personally made a small ATM finder for the Bank I worked at last, which I thought was cool, but it didn't generate any interest at the time. Internal to organizations I've heard of a number of cool apps... A small delivery company used the bot, integrated with SMS through Mobile MSN, to inform drivers of new pickups when they were out automatically. A college in Nebraska uses it in their PM department to inform employees when status reports or deliverables are due on projects. And one of the coolest I've heard of, a guy made an AIM bot for his office that had menus for all the local restaurants. Around lunch time, the bot would prompt everyone in the office to vote for their preference that day, and then would randomly pick someone to go make the orders and run to get them.

Lastly, the closest thing I've seen to a killer app... this bot integrates with a blog comment system. Essentially, if you comment on a post, it will tell you, by IM, when someone comments after you. To post another comment, just reply to the IM session. Easy, and uses the power of IM to leverage the blog. You can surf away from the page and not lose the conversation. Cool!

Dr. Ralph Kimball, Rock Star

I was fortunate enough to receive an invite from Informatica to attend a sminar in Atlanta by Ralph Kimball titled Data Warehousing:
Best Practices in Today's IT Environment. The event was packed and I was looking forward to it as a great intro for a new sales woman that is working with my company.

It was a good seminar heavy on the basics of data warehousing, including dimension design and what-is-a-fact kind of information. Dr. Kimball stressed that executing the basics correctly is the foundation for any successful implementation. He also had a few general points to always keep in mind. Among them:

• Start with what the end user wants or needs to see, ie the front end. Work backwards from the reports, graphs, or pie charts they want to see.
  
• Everything distills into 3 kinds of grains: transactions, periodic snapshots, and accumulating snapshots. Don't try to do anything fancier.
  
• Don't start by trying to identify Facts. Know the process, grain level, and deimensions first, then get the fact.
  

It's good to get a reminder of these things, as it's easy to get lost in all the necessary but distracting detail of data warehouse architecture. Too often people can lose sight of the value that was promised to the end users before the project (or the current upgrade) began. It was definitely a treat getting to see Ralph Kimball in person, and if you ever get the cahnce don't miss it.

XBRL Gathers Momentum With SEC and EC Attention

XBRL Gathers Momentum With SEC and EC Attention from Bank Systems & Technology notes that XBRL (an XML format for reporting a company's financial information) is receiving alot of support as more companies adopt it. It is receiving support now from the SEC in the US and from the EC in Europe.

Aside from the advantages for stock analysts, which will be able to pull in financial reports quickly and analyze them, this has some real cool implications in the Data Warehousing world. Pulling in competitors financials may be a snap one day, giving a piece of one of the things most data warehouses are sorely missing (and have no plans to integrate, because it can only be done with much manual effort right now) - data from sources external to the company. Knowing that even though sales increased 15%, but market share fell by 2%, would be very valuable intelligence indeed for any reporting environment.

Data Warehouse Appliances: Driving the Business Intelligence Revolution

DmReview (another favorite mag of mine that's a sure chick magnet) has an interesting article about a trend to keep an eye on, Data Warehouse Appliances: Driving the Business Intelligence Revolution. It argues that a new approach of harware and software designed for data warehousing should replace the current patchwork that makes up most current environments.

In the absence of significant architectural improvements, the traditional answer to the growing BI problem is to continue to add more hardware. For example, a company may use an Oracle DBMS, an HP server and a storage solution from EMC. As their system grows, they may add Hitachi storage and a second server. These types of systems require data and user applications to be continuously tuned and optimized.

...
The data warehouse appliance is designed specifically for the streaming workload of BI and is built using commodity components. It architecturally integrates hardware, DBMS and storage into one opaque device and combines the best elements of SMP and massively parallel processing (MPP) approaches into one that allows a query to be processed in the most optimized way possible. A data warehouse appliance is architected to remove all the bottlenecks to data flow so that the only remaining limit is the disk speed - a data-flow architecture where data moves at streaming speeds.

Sounds a bit like the Teradata vision to me. It's a good argument though, sooner or later the black art of constructing the architecture of most BI environments will become much more standard over time.

Or, I can deifintely see the need for DW and BI appliances specific to a business process. Something that could (in a perfect world) get dropped in to an environment, and after data is mapped from sources to destination, it would magically have a multitude of of reports that can jump start a business unit.

Arup Nanda On Transporting Warehouse Data

From Mark Rittman's Oracle Weblog: "OTN are currently running a good article by Arup Nanda, on the ways of transporting warehouse data using Oracle 10g. Entitled 'Moving Data Faster', it takes the form of a case study and looks at Data Pump, transportable tablespaces, and something referred to as 'pulling tablespaces'. So what's 'pulling tablespaces' about? "

Read more.

OSQL Utility - or scripting for SQL Server

Database Journal has an article on SQL Server's OSQL Utility which allows you to connect and interact with SQL Server from the command line. Batch scripts for administrative tasks are something admins in a Unix environment take for granted and can't imagine being without, and with good reason... so if you're stuck in the GUI of Enterprise Manager and Query Analyzer break out of it for a bit and get a feel for running some jobs through OSQL.

Good applications for it are running jobs on multiple servers, installing new servers, and storing schema snapshots. In truth, just about any repetitive task is possible.

Open Source Database usage rockets up in Businesses

In another Intelligent Enterprise column, BI Driving Interest in Grid mentions a study by Evans Data titled "Database Developer Study" with a few interesting statistics. Leading the story is that 26% of respondents are interested in implementing some sort of Grid computing, probably from either IBM, Oracle, or Sun.

Buried further in the article, it notes 64% of respondents use open source databases in some capacity, up from just 7% three years ago. (of those, 48% use MySQL) Do CIOs realize this? Someone should give them an update.

Is there a magic tipping point when the open source option is assumed viable in the enterprise? When is it?

Intelligent Enterprise Magazine: Readers' Choice Awards

I like Intelligent Enterprise magazine. Although thin for a trade mag, all the articles are useful and relevant to real BI problems and solutions. So I would trust the readers of it in the Intelligent Enterprise Magazine: Readers' Choice Awards.

Microsoft dominated many of the categories, including OLAP (Analysis Services), Knowledge Management (Sharepoint), and B-to-B Integration (Biztalk). Suprisingly, despite it's youth, Reporting Services won the Ad Hoc Reporting category. Over 28% of business under $500 million use it, compared to only 13.6% of large enterprises, but I have a feeling that percentage will quickly rise.

Oracle still has a lock on the DBMS for Data Warehousing category (10g), and probably will for a very long time to come.

Geographic Information Systems and Business Intelligence: Perfect Together, from Information Builders

Geographic Information Systems and Business Intelligence: Perfect Together, from Information Builders is a white paper that covers the "new and exciting convergence of GIS and BI." Does a great job at covering from a high level the value the convergence provides. Also mentions GIS powerhouse ESRI.

I say it every few weeks, and I'll say it again... for a good intro to GIS capabilities, give Microsoft's Mappoint a whirl. It's not a 6 figure solution (not even a 4 figure one), and it works quite well. I used it in a demo for a major Bank on Tuesday (with the OLAP add-in), and the effects, as usual, were jaw dropping.

Pew Internet & American Life Project: Instant Messaging

Pew Internet & American Life Project: Instant Messaging is popping up in quite a few places, and what it says reinforces the AOL study a couple of weeks ago. In a nutshell, IM is mainstream, and getting even moreso every day. Lots of interesting stats in the report, and it's worth a download.

Reporting Services Article Series

Master Chart Reports: Pie Charts in Reporting Services is the start of a series of articles in database journal by MSAS Architect Bill Pearson where he will cover all the charting capabilities of Reporting Services. So get past the tabular reporting with a basket full of numbers and impress someone. I'm looking forward to this series, as RS has potential to be a robust dashboard app.

Here's a nice presentation on deploying Reporting Services in an Enterprise. Simply explained with lots of pictures.

Lastly, here's a a short in-the-wild impression of Reporting Services, and it sounds like all the others Ive come across. Haven't read a negative review yet, and they usually don't even mention the fact that it's free.

InfoWorld: Intelligence everywhere

Intelligence everywhere by Ephraim Schwartz :

If Y2K is remembered for getting companies to buy new hardware and upgrade old software, the latest driver of change, Sarbanes-Oxley, will be remembered for democratizing information and making accountability a companywide responsibility. Its reporting requirements make it mandatory that businesses hold everyone's feet to the fire.

Enter pervasive BI. Sometimes called situational BI, pervasive BI will mean that management at every level of the organization has access to intelligence and key performance indicators that are relevant and actionable. It ensures that the same kind of information is disseminated at every level down the chain -- divisional, departmental, and regional -- to a local team leader using the same dashboard interface.

I'm not sure if SOX is the real catalyst, but I do think BI is well overdue to start working it's way down to the line managers, which are the ones who can take immediate action on what they see.

Is Tableau the Next Google?

No it's not. Despite the hype (including slashdot), it's pretty much another OLAP front-end, just one getting an awful lot of press from people that haven't been exposed to multi-dimensional reporting apps before.

That said, it does look cool, and I'd like to mess around with it. Nice and clean with a drag and drop interface. Here's their pdf brochure. Only supports Microsoft products (Access, Excel, SQL Server, Analysis Services) for now.