Background

XYZ companies was built Green ICT 2010 promo website. Green ICT is a concept that usually connected with effort to reduce energy consumption and the other natural resources, beside emission and rubbish that produced from information technology and communication activity.

This website contains of XYZ Green ICT Services. There are three services offers on this website:

1. Data center: Is providing and rental of storage facilities / space / building with the standard International Data Center
2. XYZ Net: Is dedicated internet service either domestically or internationally by using fixed connection.
3. Idola Broadband: Is and economical high speed internet service. Idola Broadband is supported by the Fiber Optic Next Generation Services.

On this website people can register or apply each service using available application form.

Statement of Problems

This website launched for several month ago. But the company doesn’t know about the user experience of this website. The company hope that users whose register the registration on this website increased. Companies worry the users will be reluctant to register because they equate registration with payment. That is actually not the case because registered users are, in fact, allowed to cancel.

Proposed / Goal of Study

Company attempts to alleviate the problem by giving a notification in which the offer is being displayed explaining that users are permitted to cancel even after they have registered.

The premise is that if the notification is displayed and is read by users, they should be less hesitant to apply because they now know (from the notification) that they can cancel their order and they won’t be asked to pay anything. Therefore, we should see an increase number of registrants from segment that sees the notification.

Benefit of Study

In web development, to create a new web design must based on user experience, but it’s difficult to know the user experience on the website. This study, will allow companies to know about user experience on the website, improve the conversion rates to reach goal (for business purpose), Decrease visitor bounce rates, increase visitor satisfaction, and eliminate guesswork from site design.

Hypothesis

The segment that sees notification converts better than the segment that does not.

IV: The segment that sees notification

DV: conversion rates (Conversion rates = Number of Goal Achievements / Visit)

Scope and Significance

This research will take XYZ’s micro site. This research will only specific on Idola Broadband’s page of XYZ’s micro site. There are two treatments that planned to replace the current version of Idola Broadband’s page. Both of them have different notifications to convince users. Then the experiment test will be conducted for 2 weeks.

Conversion Rates

According to Kohavi, Henne & Sommerfield (2007), “Conversion rates is the percentage of visit to the website that include a purchase”. In this case of research conversion rates will be percentage of visit to the website that include a register. Conversion rates is defined as follows:

Conversion Rates = Number of Goals Achievement / Visits

There are some factors that can be use to conversion rate, including:

• Use a credibility-based professional design A professionally designed site makes the type of first impression (fast, mistake-free, attractive, and credible). Accordingly, recent studies in the web filed, concluded that users form their first impression of the web sites based on overall visual appearance, rather than on a particular element (Schenkamn & Jönsson, 2000).

• Make website navigation easy Design should allow visitors to get where they want to go quickly and easily with a minimum number of clicks.

• Optimize the credibility of your logo Haig (2007) found that a logo which contained the credibility traits of a website company induced 2x to 4x more click-through than logos which did not have the same credibility traits and were thus considered non-credible.

• Write a memorable slogan The slogan should be communicate the biggest benefit that your product provides, simple yet memorable, use active voice with adverb near the verb for more impact, differentiate brand, designed for future expansion, embrace ambiguity, prime desired attributes, jump-start recall with jingles.

• Use benefit-oriented headlines Use headlines that clearly state the most important benefits that your product or service offers. For example, emphasize saving money, time, and energy.

• Give important content the best placement The position of components on web pages can make a significant difference in your website conversion and site flow through. Chen, Rose and Bederson (2009) agree with Jakob Nielsen’s eye tracking visualization that users look first at the top-left corner of your web page and scan to right and then to the left in an F-shaped pattern.

• Include appealing offers and calls to action Asking visitor to act: to purchase, sign up, or opt in.

• Deploy persuasive, benefit-oriented content Avoid feature-oriented copy and use bullets and value hierarchies. The benefit bullet format presents the benefits of product in the order of its value hierarchy to your target market.

• Use illustrative product and service Images The images that you display on your web pages can significantly improve conversion rates.

• Use interactive elements to engage users Immediately interest and engage site visitors with interactive website components. The elements invite visitors to focus their attention on message. They include audio, video, and web-based devices such as Flash movies and interactive customer support tools such as LivePerson.

Experimental and Testing

Experimental and testing needed to know the feedback from web visitors. According to Waisberg and Kaushik (2009), “The web analyst must try endlessly and learn to be wrong quickly, learn to test everything and understand that the customer should choose, not designer or website manager”.  The most basic tuning method available is A/B split testing. The names comes from the fact that two version of landing pages. “Split testing” refers to the random assignment of new visitors to the version of the page that they see.

Baccot, Choudary, Grigoras, and Charvillat (2009) also agree that the researcher often use A/B test to carry out experiments on the web. A/B experiment test (split testing) involve testing one page against one or more alternate pages. The alternate page should have point of different with the current page.  Each version of the page has its own URL. For example:

1. http://www.mysite.com/landing_page.html (Version A, also known as the original or control)

2. http://www.mysite.com/test_page_b.html (Version B)

3. http://www.mysite.com/test_page_c.html (Version C)

Picture 1: High-flow for an A/B test

With an A/B experiment testing two complete separate pages. So web designer could design a page with an entirely different layout, navigation. With this test can also test smaller changes like new copy, headlines, and images.

Study Design

This research will need to create two new design from Idola Broadband version page that assumed can improve the conversion rates. Each new version of Idola broadband page must be different and they will be the treatments and the current version is control page. This new version with the highest conversion rates will replace the current version.

1. a. Control Page

Control page is the current version of Idola Broadband page. This current version is used as a benchmark of how large the increase conversion rates from the lastest versions or treatment page.

Picture 2. Current version of Idola Broadband page (Control Page)

In this current version there are no notification in which the offer is being displayed explaining that users are permitted to cancel after they have registered.

1. b. Treatment Page

The treatment page is new version of Idola broadband pages. There are two version that called Idola 2 and Idola 3. The difference between the control page and the treatment page is located at the notification that informing the registration can be canceled at any time.

Picture 3. Treatment page of Idola Broadband Page (Idola 2)

In Idola 2 have three notifications are limited time promotion, apply immediately to be contacted XYZ, and the cancellation of the application can be done anytime.

Picture 4. Treatment page of Idola Broadband Page (Idola 3)

It has little different with Idola 2, in Idola 3 only have two notifications and do not have the notification regarding the application of time limits.

Data Collection Method

Each visitor who visit this website will be tracked using Google Analytic tools and the experiment test will use Google Optimizer tools. Some Google analytic and Google Optimizer scripts embedded on each action and page on  XYZ’s website. Google Analytics will be collected web analytics data after visitors visit the website and have some experience on the website.  There are some web analytics data that can be collected using Google Analytics and used on this study:

-       Date, describing the time when the data collected.

-       All Visit, the number of visitors overall website.

-       Page Visit, the number of visitors on specific page.

-       Goal, the number of goals obtained.

First time Google Optimizer will detect new visitor then it generates random Idola page version for new visitor. Even new visitors are detected by Google Optimizer script, the new visitor will save on Google Optimizer tools as old visitor. Google Optimizer saved which version page of Idola Broadband that generate for each known visitor.  For example, if visitor get page A then this visitor will never get B. Each visitor for this website have unique key on Google, so when visitor come to Idola page for twice this visitor will identified as known visitor and always get page A.

<!-- Google Website Optimizer Control Script -->

<script> function utmx_section(){}function utmx(){}(function(){var k=’2371568844′,d=document,l=d.location,c=d.cookie;function f(n){if(c){var i=c.indexOf(n+’=');if(i>-1){var j=c.indexOf(‘;’,i);return c.substring(i+n.length+1,j<0?c.length:j)}}}var x=f(‘__utmx’),xx=f(‘__utmxx’),h=l.hash;d.write(‘<sc’+'ript src=”‘+’http’+(l.protocol==’https:’?'s://ssl’:'://www’)+’.google-analytics.com’+'/siteopt.js?v=1&utmxkey=’+k+’&utmx=’+(x?x:”)+’&utmxx=’+(xx?xx:”)+’&utmxtime=’+new Date().valueOf()+(h?’&utmxhash=’+escape(h.substr(1)):”)+‘” type=”text/javascript” charset=”utf-8″></sc’+'ript>’)})();</script><script>utmx(“url”,’A/B’);</script>

<!-- End of Google Website Optimizer Control Script -->

Script1. Google Optimizer Script to Control The Experiment

The goal of this campaign is visitors who apply Idola Brodband application form, so need to put Google Analytic script on submit action.  Google Analytic will catch and collected the data of visitors and version of Idola Broadband pages that can reach the goal.

Data collected will only for human or not robot. Robot will automatically excluded by default on Google Analytics. Robot is an automatic web crawler from search engines that visit every page on the Internet. The experiment will not have valid result if robots are included on this data collection.

Population

XYZ promote this web campaign through Kompas media. This will be assumed that visitors who come through Kompas media does not recognized the services of the previous XYZ. In contrast to the visitors who come through the XYZ’s company website or visitors who previously had known XYZ’s services. All visitors who visit the XYZ’s website through Kompas will be the population in this experiment. Population on this research are not limited to age or gender.

Target of XYZ Idola broadband service are individuals or companies. When someone is interested in opening XYZ’s campaign website and visit the website, the that person will become participants in this experiments. The population to this experiment on this research is  human only. Robot or automated web crawler from search engines are excluded by default by Google Analytics.

Sample and sampling technique

The key of the sample is random, user cannot be distributed and no factor can influence the decision. The probability on control page and each treatment is 33,3% of 100%.  For example if all population on this research is 300 visitors then the visitors for each version of page will be 100 visitors for control page, 100 visitors for Idola 2 treatment, and 100 visitors for Idola 3 treatment. The count of visitors for each version page on this sample is not exactly 100 visitors because the probability for each version is 33,3%.

Experiment data on this research will be counted from August 2^nd to August 17^th.  It’s due of  XYZ want to improve more conversion rates quickly. The sample of this research will only for visitors who come to XYZ campaign through Kompas only and it’s assumed that visitors don’t know or familiar about XYZ’s services.

Participants

The XYZ Green ICT promo website have been promoted trough Kompas from August 2^nd to August 17^th. At this time, the experiment test is running and the user experience from the website is collected using Google Analytic. There are 3151 unique visitors that came to XYZ Green ICT Promo website through Kompas’s ads link between this duration time and the average unique visitors is about 210 visitors per day.

Picture 5. Participant Graphic

From all unique visitors, there are only 771 unique visitors came to Idola Broadband page and the average unique visitors to Idola Broadband page is about 51 visitors per day. All unique visitors that visit Idola Broadband page will separately divide by random to 3 version of Idola Broadband page. The participants can’t choose which version of Idola Broadband page that they visit. The participants that involved in this experiment can be came from anywhere and not limited by background, position, gender and age.

Descriptive Analysis of Research Findings

The data that collected from Google Analytic is calculated using statistical software SPSS 18, the conversion rates for each treatment is calculated and compared with the current version of Idola Broadband using Independent-Samples T Test on SPSS. Each new version is grouped in one group and two treatment groups are compared to control group.  For example current version (Idola) is compared with the new version (Idola 2), then Idola will be group A and Idola 2 will be group B. The result from the experiment test is shown in table 4.1 and table 4.2.

Table 1. Result of Experimental Test Between Idola and Idola2

From the table above can be seen that the new version of Idola Broadband page that has notifications and not mention time limit (Idola2) is significantly increase the conversion rates.

Table 2. Result of Experimental Test Between Idola and Idola3

From the table above can be seen that the new version of Idola broadband page that has notifications and not mention time limit (Idola 3) is significantly increase the conversion rates.

The result from the experimental test showed that both of treatment pages are significantly increase in conversion rates and the hypothesis is accepted.  Both of treatments page can be used to replace the current version of Idola Broadband page. From the treatment pages of Idola Broadband pages can be seen that the Idola2, which mention the time limits have significantly higher levels compared with Idola 3, which did not mention a time limit.

Rationale of Study

This research needs to be done to test a new website page whether it can replace the old pages. New version pages are not necessarily to make this campaign to obtain better result. In accordance with the topics on this research, the current version pages are still considered to be confusing visitors, giving rise to doubts visitors to register in this campaign. Expected that with the new version of a web page that contained a notification message can reduce user’s hesitation on web campaign.

Result

The result by the experimental test on this research is accepted the hypothesis. The two treatment pages for Idola Broadband can significantly increase conversion rates better than the previous version. It can be concluded that by adding notifications on Idola Broadband page on this web campaign can make visitors feel more confident to do the registration. By adding a time limit on notifications will increase more significantly compared with not mentioning the time limit.

Discussion

In this study has been discussed about how to reduce user’s hesitation on web campaign by adding a notification on the new version of web campaign page. This study used experimental AB test with a randomized trial gave different versions of web page to visitors within two weeks.  The result shown that the segment that sees notifications converts better than the segment that does not.

Actually, in conducting experimental AB test within web campaign, the longer duration time for data collection will make the better result to know significantly increase the conversion rates. It’s caused the experimental test gave visitors the random version of page and the result is significant if the conversion rates is not caused by random. Because of participant on this study only visitors who came to this web campaign through Kompas’s Ads link and assumption no bias, the data for experimental test not enough to collected. It’s caused by the company only have a budget for two weeks advertising.

Limitation of Study

This study only discussed about reducing user’s hesitation on web campaign by adding notification on new version of web campaign. It’s just small component that can be added to increase the conversion rates. There are many components that need to discuss to reach the goal on web campaign. The other researcher research using color segmentation and web page layout, but there are many more other components that need attention such as segmentation on visitors, adding interactive video, landing page iteration, social media tracking, etc.

Recommendation of Study

This study has been carried out research to reduce user’s hesitation on web campaign based on the addition of notification on the new versions. This study is not notice visitors by age or gender, whereas gender and age can be determine the acquisition goals in campaign. For further research, user segmentations by age and gender can be added to know the better conversions rates from each user segments. Then by focusing on the marketing target of web campaign, web design can be exactly correct message to the markets. For example the company planned to create web campaign about beauty contest. Then the visitors are girls between 18 to 21 years old. The web designer need to doing some research to know what are favorite by the target.

Summary

This study is talking about reducing user’s hesitation on web campaign. On web campaign, design and user usability is one of important component to reach campaign’s goals. The XYZ Company feels that the current design on Idola Broadband page is not capable in achieving more goal, they want to replace the current design or version with the new versions. To replace current design to new design, web designer must know about user experience within the current and new version. It’s caused by new versions not always will increase the conversion rates to reach web campaign’s goals. Sometimes, a design can’t properly convey messages to users and need to add notification to design.

On this study, the notification is added to new versions. It’s expected can be reduced user’s hesitation on web campaign. Then the current version and new version will go live together using experimental AB test. The users experience within current and new version can be collected to proved that segments who sees notifications will converts better.

Refference

1. Alt, Brian and Usborne, Nick. Marketing Experiments Journal. [Online] December 29, 2005. http://www.marketingexperiments.com/improving-website-conversion/multivariable-testing.html
2. Alsudani, F. & Casey, M. (2009). The Effect of Aesthetics on Web Credibility. The British Computer Society Journal.
3. Baccot, B., Choudary, O., Grigoras, R.,  & Charvillat, V. (2009). On The Impact of Sequence and Time in Rich Media Advertising. Portal ACM.
4. Chen, R., Rose, A., & Bederson, B. B. (2009). How People Read Books Online: Mining and Visualizing Web Logs for Use Information. University of Maryland Journal.
5. Haigh, W. L. (2006). How and Why Credibility – Based Company Logos Are Effective in Marketing Communication In Persuading Customers To Take Action: A Multiple Case Study Toward a Better Understanding of Creativity in Branding. Southern Cross University.
6. King, Andrew B. (2008). Web Optimization, Page 300, 305. OReilly, USA.
7. Kohavi, R., Henne, R. M., & Sommerfield, D. (2007). Practical Guide to Controlled Experiment on the Web: Listen to Your Customers not to the HiPPO. Portal ACM.
8. The MarketingExperiments Quarterly Research Journal. Volume I, Issue 2 – Q2 2010. [Online]  August 7, 2010  http://www.marketingexperiments.com/marketing-optimization/q22010.html
9. Schenkman B. N. & Jonsson (2000). Aesthetics and Preferences of Web Pages. Behavior & Information Technology, vol. 19, No. 5, 367-377
10. Waisberg, D. & Kaushik, A. (2009). Web Analytics 2.0: Emprowering Customer Centricity – Part II. SEMJ Journal.