COMPARISON

10 points to consider when comparing ATHENA vs. developing a system from scratch

   click here to download the MS Word version of this page

1) Getting Exactly What You Want
As counter-intuitive as it seems, ATHENA is more likely to get your company “exactly what it wants” sooner than an in-house development team. SurfMerchants has years of experience building a flexible core engine to support the myriad of functions that a comprehensive mystery shopping system requires and also has successfully accommodated the requests of over 20 mystery shopping companies over that time and has never lost a customer.

An in-house team will have to go through several trial and error phases to achieve precisely what your company requires and will most likely need to rework their initial version entirely after the first few clients with unpredicted needs find holes in the system (as SASSIE had to).

After dealing with over 300 end-clients requests and undergoing two major revisions, SASSIE’s core structure is mature and already has multiple layers of flexibility installed, many of which only become apparent when a client with unusual configuration tests its limits. Add in ATHENA’s freedom of development and you have a system poised to move in any direction your company requires.

To be honest, there is no such thing as permanently getting “exactly what you want”. From what we’ve seen in this industry, the real competitive factor is getting your clients “exactly what they want”.

2) Training (if your company is currently using SASSIE™)
Retraining staff to a new system is difficult, especially if they perceive a move from a reliable, user-tested system to a system that is a work in progress at their expense. Moving shoppers from a system they are familiar and productive in (especially considering that there are hundreds of thousands of pre-trained SASSIE shoppers from other SASSIE accounts) can result in many months of confusion, heavy shopper support loads, and improperly completed surveys. Finally, to move end-clients and require them to re-learn a new system with features they have come to rely on is often a costly and time-consuming effort that has sometimes resulted in clients moving to other SASSIE-using providers in the past.

3) Existing Data (if your company is currently using SASSIE™)
Moving SASSIE data to an ATHENA system will be seamless and nearly instantaneous. Moving SASSIE data to a new system is certainly possible, but comes at some expense and there is no guarantee that a new system will accept the export without some translation utilities, which are costly and difficult to engineer.

4) Cost Effectiveness
ATHENA is more cost effective to develop and maintain than keeping an in-house staff on salary.

ATHENA’s costs can be easily predicted and budgeted for, whereas the final cost of launching a large scale application is always unknown. With the vast majority of code already written and tested, purchasing ATHENA’s core engine is an extremely cost-effective option – your company will essentially be getting hundreds of thousands of dollars of code at an extreme discount.
With an array of licensing options, maintaining an ATHENA system over time is also very cost effective. Whereas maintaining a support an development team in-house over time is extremely expensive, your company can benefit from ATHENA/SASSIE’s economies of scale and pay for only what it requires AND rest easy with the knowledge that additional resources are readily available when needed (there is no guarantee that a laid off employee or an engineer re-assigned to other tasks of divisions within Your company will be available upon demand).

5) The Think Tank benefit
SurfMerchants’ wide experience in mystery shopping with thousands of surveys and hundreds of end clients can prove invaluable in finding quick solutions to complicated issues. We can alert your company to upcoming problems, advise on current mystery shopping trends, and suggest various add-on products, strategies and technologies that have proven popular and successful in the past. Our capacity to consult on this industry cannot be replicated with an in-house development team.

6) Time To Launch
Historically, companies have severely underestimated the amount of time it takes to build a comprehensive mystery shopping system from scratch – the industry average for a comprehensive system is somewhere between 12-18 months. This time may prove crucial in 2003 as a study commissioned by SurfMerchants predicts that within the next two years there will be consolidation of the high end of the market into fewer and fewer major players (not coincidentally, this study confirmed SurfMerchant’s desire to offer ATHENA as an alternative to SASSIE!).

Over a third of current SASSIE subscribers abandoned development on their in house systems and migrated to SASSIE. Several companies thought to use SASSIE as a temporary solution until they completed work on their in-house system, only to find that their clients’ reliance on SASSIE’s feature set (and the time it would take to replicate the features in-house) made their development efforts obsolete.
The primary reason that mystery shopping systems take longer than predicted to develop is that that project managers neglect to factor into the development time of some non-obvious features that every high-volume mystery shopping system requires.

Here are some examples of non-obvious features, followed by the number of employee-weeks it took SurfMerchants to conceive, program, test, revise and perfect these capabilities:

• Form generation tools (3 weeks) – it’s no trick to have a programmer create forms one by one, but to have tools that allow non-programmers to create complex forms with a sufficient array of question types and scoring options is a daunting task.
  
• Programmable custom scoring engine (4 weeks) – establishing standard scoring rules is easy to program, but buliding a scoring engine flexible enough to allow custom scoring of nearly unlimited complexity.
  
• Bulk PDF report generation (6 weeks) – building a tools for clients to generate a single survey PDF is fairly easy, but a majority of clients want an option to create compilations (often of hundreds to thousands of surveys) that can be printed in bulk. And unless it’s acceptable to have your staff devoting valuable time to generating these PDF’s for impatient clients, these tools should be usable by clients using only a web interface.
  
• Automated Scheduling via Email (6 weeks) – Determining which shoppers to send which shop offers is easy enough, but building a system capable of caching and distributing up to hundreds of thousands of emails a day without getting spam filtered is a much higher level of programming. Standard mailing list servers are all but useless here since these programs would send every shop offer to every shopper on the list instead of emails targeted to shoppers with the correct demographics and correct geographical location.
  
• Shopper Billing system with PayPal™ compatibility (4 weeks) – Shopper Payment is one of the great hidden productivity killers in mystery shopping, and a powerful method to track shop fees vs. reimbursements is a must. Since many MSPs are moving to PaypPal as an extremely time and cost effective method to pay shoppers, having a system that can interface with PayPal bulk payments is an important capability.
  
• Automated Report distribution (3 weeks) – many clients’ managers do not visit the web sites often enough to know when new shops relevant to them have been posted on the system, so an automated method of contacting each manager and giving them one click access to their reports is crucial.
  
• Advanced User Access system (7 weeks) – The majority of clients want their managers to have restricted viewing access on the system – for instance some may want district managers to be only able to view results for locations under their management, where others may want district managers to be able to view information for all districts. This requires that every shop log and report recognize each manager’s access level and present a customized view for each manager.

Just from viewing this partial list, it’s easy to see how production schedules can spiral out of control due to features and functions whose necessity is hard to predict.

7) Focusing on Core Strengths
In our business of web development, it is easy to be tempted to branch out into areas beyond our expertise (i.e. marketing, multimedia, web strategy consulting). However, we understand that one of the main contributors in out success is our intense focus on doing what we do best: building web applications. By focusing on our core competencies, we were able to remain successful during a time when our less focused rivals were going out of business – we did not try to be all things to all people.

Likewise, your company is faced with a decision on whether it wants to be a mystery shopping company or a mystery shopping / programming company. Both are worthy goals, but I would suggest that your company’s growth will be much faster if it focuses on the marketing, selling, and execution of mystery shopping services, rather than being spread more thinly by its other identity as a programming/development company.

8) Risk
The value of a thoroughly battle tested & debugged system can be immense in our industry. It only takes a few mistakes (i.e. a calculation error in a report, an improperly programmed form requiring an embarrassing series of re-shops, a server crash that loses data, a security breach that exposes client information or shopper personal data) for a mystery shopping company to lose clients, or worse, develop a poor reputation in so small an industry. The chances of quickly building a in-house system with no crucial missteps at any juncture are quite slim.

9) Infrastructure
One of our unexpectedly difficult challenges in building SASSIE and ATHENA was building a technology infrastructure capable of handling a heavy volume of shops from any point in the world on a 24/7 basis. It was not as simple as buying an expensive set of servers and installing industry standard software on them – in fact it is only through massive trial and error, constant fine-tuning and difficult infrastructure moves that we arrived at a stable architecture. Even so, in anticipation of even higher shop volumes due to ATHENA we are peremptorily launching a $50,000 infrastructure upgrade this December. Here is the list of major changes we have undergone in order to keep our infrastructure up to the demands of our clients.

• 4 hosting providers
• 6 different server hardware configurations
• 2 operating systems (each customized heavily to SASSIE specifications)
• 8 hardware upgrades
• 35 changes to the database and server settings (fine tuning)

When constructing a new system from scratch, it is important to remember to factor in the time it takes to determine a stable architecture in addition to the time it takes to program the system.

10) The “Secret Weapon” factor
One factor that gets little attention is the large amount of small utilities that need to be created for our support team to execute data changes for mystery shopping companies and their clients on demand. No one other than our staff ever sees these utilities and they would probably never show up on a project development plan if one were building a system from scratch. However, the value of these utilities and their instant accessibility are tremendous. Here are just a few examples:

• Bulk Score Recalculator
• Manager Synchronizer – alter shops within a date range to reflect current manager setup
• Manager Password Mailer
• Admin activity logging
• Backup Retrieval utilities