The Contact Analysis Checklist

Prior to constructing your flows, it’s a good idea to conduct a contact analysis to learn more about your intended audience. We’ve put together a checklist containing questions to help you analyze your contacts and ensure efficient and pleasant communication. This checklist can be applied at any point in the testing process.

The Checklist

  1. How many contacts do you anticipate will interact with your service ?

  2. What information do you have regarding the demographic makeup of your contacts (age, gender, education level, ethnicity), and how might you use that information to develop and shape your flows? Will it be necessary to build a multi-language flow?

  3. What personal and professional traits do you have in common with your contacts?

  4. Are there any cultural considerations that may influence how your contacts interact with your flows?

  5. Will you need to targeting certain contacts? If so, what is your criteria?

  6. With regard to your service, how much knowledge or expertise will your contacts possess?

  7. Will your contacts recognize and implicitly trust your credibility or ethos, or will you have to earn their trust by immediately demonstrating value?

  8. What preconceptions or biases regarding your service and/or channel might be held by some of your contacts?

  9. In general, will your contacts have a favorable, unfavorable, or neutral attitude toward the subject and/or nature of your service?

  10. What common misconceptions about the subject of your service will you need to correct during the course of your deployment?

  11. In general, will your contacts recognize the value of your service, or will you need to generate interest and demonstrate value?

  12. What expectations might your contacts have regarding the length of your flows?

  13. What expectations might your contacts have regarding content of your flows?

  14. To complete your flows, how much background information will your contacts need?

  15. What key questions might your contacts have about your service?

  16. Which tone or general attitude do you want to convey to your contacts - objective? authoritative? collaborative? supportive?

  17. What do you want your contacts to know or do as a result of interacting with your flows?

Feature Update - Simplified Expressions

Writing expressions in TextIt just got easier :)

In an effort to simplify the process of adding expressions to flows, we’ve changed the input method from = to @ and added a completion dialogue that lists and explains each function when @( is written in any window:  

1png

Variable substitution remains the same, so @contact and @contact.name are unchanged. 

With these new changes, what was previously written as =UPPER(contact.name) or @contact.name|upper is now written @(UPPER(contact.name))

To escape the variable and function dialogue, simply add an additional @ to the text. For example, @@twitter will evaluate to @twitter when a message is sent. 

Refer to our function glossary for a list of functions you can add to your messages. 

We’re excited to get your feedback; share your thoughts with us at support@textit.in.

All the best,

The TextIt Team


Use TextIt to funnel your users to live support

While automation is useful in a variety of contexts, there may come a point in the interaction when it makes sense to add a human element - to link a user to a team member who can continue the conversation in real time. This would be an obvious win in a customer support context, but extends across any vertical to organizations that want to improve communication with a particular community.

Create an SMS-to-Support Pipeline

The key to this configuration is TextIt’s Send an SMS to somebody else action. You can place this action at any step in a flow, and use it to instantly direct contacts to team members who can respond via SMS, phone or email to continue the thread.

Create an Email-to-Support Pipeline

If your organization has multiple customers or manages multiple members, you’re probably using a customer relationship management platform (CRM). If you’re not, you should be. CRMs allow you to track and manage the people that interact with your product or service, and most offer the ability to link your organization’s email addresses to a feed that each team member can view. CRMs built for customer support, like UserVoice, present these emails as tickets that each team member can respond to and eliminate from the feed.

Another super simple way to connect a contacts’ inquiry with a team member is the Send an Email action. Simply create a flow that asks for a contact’s name, inquiry and preferred response method, and configure TextIt to send an email containing that info off to your support feed. Team members will be able to instantly access the ticket via your CRM and respond accordingly. Easy.

1png

The variable @flow.name references the name of the contact if you collected it at a previous step, while @flow.inquiry and @flow.preferred_channel reference the contact’s inquiry and the way they’d like you to respond (SMS, Call, Email), respectively.

Tiered Support

Perhaps you’re an organization that tiers your support to accommodate low support capacity, or you want to screen a customer or community member before directing them to a team member. No problem. Decide on your requirements, then build a flow that only directs contacts to staff if they follow a certain path in your flow.

Create a Flow, Build a Stream

Support isn’t complicated, but it is vital to the success of an organization. Use TextIt to provide a convenient, mobile support channel, and respond with a human touch.

Learn how Bring uses TextIt to calculate its Net Promoter Score

Bring, a Norway Post brand, is one of the largest postal and logistics companies in Scandinavia. Bring uses TextIt to measure customer satisfaction after each delivery. Since signing up in March, they’ve collected SMS feedback from over 5,000 of their Express customers in Norway.  

Says Bring: “For us it’s important not only to think about the distribution of goods to the world, but also to make every customer satisfied, whether they’re small or big.” Bring employs a number of methods to measure customer satisfaction, SMS being the most convenient.

How it works

After each delivery, the package recipient receives a personalized SMS message from Bring that seeks to establish their Net Promoter Score, followed by a request for a reason (1 for delivery, 2 for driver behavior, 3 for condition of the goods, and 4 for service performed as promised”) and a comment to provide more insight into the customer’s rating. They then follow up with customers to learn more.

Net Promoter Score

Net Promoter Score is a customer loyalty metric that can be used to gauge the quality of a service's customer relationships. Proponents believe it’s a key indicator of growth. It’s calculated through responses to a single question:

How likely is it that you would recommend us to a friend or colleague?

Respondents are asked to rate the likelihood on a scale of 1 (extremely unlikely) to 10 (extremely likely).

Promoters

Those who respond with a score of 9 or 10 are called Promoters. Promoters are likely to exhibit value-creating behaviors like purchasing more, purchasing again, remaining customers for longer, and making positive referrals to other potential customers.

Passives

Those who respond with a score of 7 or 8 are called Passives. Passives are satisfied but unenthusiastic customers who are likely vulnerable to competitive offerings.

Detractors

Those who respond with a score of 6 or below are called Detractors. Detractors are are unhappy customers who can damage your brand and impede growth through negative word-of-mouth.

Calculating your score

The Net Promoter Score is obtained by subtracting the percentage of Detractors from the percentage of Promoters. A company’s score can range from a low of -100 (if every customer is aDetractor) to a high of 100 (if every customer is a Promoter).

Closing the loop

What good would this information be if it wasn’t actionable? Services who seek to learn their Net Promoter Score are encouraged to follow up with customers who’ve provided feedback to learn more, and attempt to change a Detractor into a Promoter.

TextIt makes this process easy, allowing you to send emails to your customer support team containing the names and contact information of Promoters, Passives and Detractors.

Sample survey

Use this sample survey to find your Net Promoter Score and assess brand loyalty:


In the example above, you'll note that TextIt's flow logic allows you to collect a customer's rating and a comment about your service, send this information to your customer support CRM via email, then direct them to a coupon located on your site in 3 messages. The conversation flows naturally, and no two conversations are the same. 

Here's an example of how that email might look: 

Use TextIt to find your Net Promoter Score

Bring recognizes the importance of establishing a convenient, direct line of communication with customers. What’s your Net Promoter Score? How do you compare to competitors? Do customers feel like you offer value and quality? Use TextIt answer these questions and more.

Click here to sign up for a free account, no credit card required. Got a question? Contact us at support@textit.in - you're guaranteed a quick response. 

Are your Flows user friendly? Find out with Usability Testing

Why conduct a usability test?

Now that you've chosen the perfect flow, you can conduct a usability test to make sure it’s provides a good experience for your contacts. Usability tests encourage you to focus on your contacts’ experience by assessing ease of use and learnability. This step is important - it allows you to make the best possible impression when you send out the final version of your flow. 

Usability test requirements

  1. A large group of test contacts representative of your target population.
  2. A clear and precise protocol for your test contacts to follow. This might include a pre-test questionnaire, the test itself, and a post-test questionnaire.
  3. A point of contact who is trained to offer instructions and answer questions. In your pilot test, the facilitator took a more hands-on approach. In your usability test, the facilitator will take a supporting role while your test contacts take the lead. 
  4. Observants who will track the behavior of the test contacts.

Elements of a usability test plan

Subjective metrics

Background questions. Ask these questions prior to the test.

Ease of use, comfort and satisfaction. Ask these questions after each task is completed.

Overall ease of use, comfort, and satisfaction. Ask when the test is completed.

Likes, dislikes and recommendations. Ask test contacts what they liked most about your service, what they liked least about your service, and if they have recommendations for improving it.

Continued use. The likelihood your test contacts will use your service when the test is completed.

Net promoter score. On a scale that ranges from extremely unlikely to extremely likely, measure the likelihood that your test contacts would recommend your service to (1) a family member (2) a friend and (3) a colleague.

Quantitative metrics

Flow completion. The flow is completed when a contact has successfully passed through each step - be it an action or split step.  

Critical errors. Critical errors result in your contacts being unable to finish a flow. For example, consistently responding with the wrong answer, an incorrectly formatted answer, an answer you haven’t accounted for, or an opt-out.

Non-critical errors. Non-critical errors are errors that the contact can recover from. The most common error you’ll experience is a response that is categorized as “other,” causing the contact to be redirected to a split step. These errors result in the flow being completed less efficiently. You’ll want to keep a tally of the amount of non-critical errors occur per contact, per contact per flow, per flow, and overall.

Error-free rate. The error-free rate is the percentage of contacts who complete the flow(s) without experiencing or committing any errors.

Time on task. The amount of time it takes the contact to complete the flow(s) - the time a flow was completed minus the time it was started. 

Test Results  

Construct an analysis of what you’ve observed. Isolate the flows that had the highest and lowest completion rates. If possible, include a summary of the completion rates by contact, step, and flow. Depending on the metrics you collected you may want to show the:

  • Number and percent of contacts who completed each flow or series of flows.

  • Average time taken to complete each flow or series of flows for those who completed them.

  • Ease of use, comfort, and satisfaction results.

  • Include your contacts’ comments if they are illustrative.

Findings and Recommendations

List your findings and recommendations using all your data. Each finding should have a basis in what occurred during the test.  You’ll want to sort findings and recommendations by flow. Finally, it’s important to keep in mind that identifying positives is equally as important as identifying negatives - what’s working well should be maintained moving forward.

Implement and Retest

For a usability test to have value, you’ll need to use what you’ve learned to improve your service. You may not be able to implement all recommendations or requests - developing anything is a series of trade-offs in which you balance a number of factors, such as budget, timeframe, and a handful of external factors.  If you can’t implement all the recommendations, assign priorities based on the most common and prevalent problems. As you prioritize, push to get the changes your contacts want.

The Importance of a Pilot Testing

Why conduct a pilot test? 

After testing your flows in the simulator, you’re ready to move to the second step in our recommended testing protocol: the pilot test. A pilot test, or "pilot," is your first trial run; a small-scale version of your larger project, and arguably the most important step in testing your SMS program. Your SMS program will be an automated system comprised of multiple components (contacts, phones, carriers, channels and flows). Moreover, it represents your organization - so it's best to test each one thoroughly.

A pilot allows you to:

  • Make sure messages are being delivered to each major carrier in your country.

  • Get a good sense of how long it will take your channel or carrier to deliver and receive messages.

  • Provide your team test-facilitation practice.

  • Evaluate the clarity of your questions and flow logic from the perspectives of your test contacts. Are they making sense to your test contacts?

  • Make last minute adjustments (carrier, connection method, flow structure/content, etc.)

  • Determine whether you’re ready to increase scale.

Pilot requirements

  • A group of 5-10 independent test contacts that represent your target population.

  • The current version(s) of your flow(s).

  • One or more pilot facilitators. This person will conduct the pre and post-pilot evaluations as well as the test itself.  

  • Observants. These people will observe the test contacts’ responses through the TextIt dashboard in addition to their overall behavior. See below.

During a pilot, it’s best to be present with your test contacts. If your flows are scheduled using a campaign, or you want to allow your contacts to respond asynchronously (throughout the day/week at their own leisure), you might compensate by communicating with them at the start or end of each day. Run the pilot 3-5 days prior to your usability test so that you have time to deal with any technical issues and/or make scenario/materials changes.

Things to look for:

  • Do the test contacts understand the objective of the flow/campaign?

  • Do the test contacts feel comfortable responding to your questions and/or performing your tasks?

  • Is the wording of your flow(s) clear?

  • Are your contacts being sorted into the right groups? Could you do a better job of sorting them?

  • Are you properly categorizing responses?

  • Are the answer choices compatible with the test contacts’ experiences?

  • Do any of the items require them to think too long or hard before responding? If so, which ones?

  • Do any steps produce irritation or confusion?

  • Which steps are receiving the most “other” responses?

  • Do the answers collected satisfy your objectives?

  • Are your flows too long?

  • According to your test contacts, has anything been overlooked?

Best practices

Remain neutral. If the participant asks a question, reply “What’s your best guess?”

Don’t lead your test contacts. If a test contact gives up, you’ll need to decide whether to provide a hint or end the test.

Focus your observations. Give productive and unproductive paths equal attention. Observants should focus on what the test contacts did in as much detail as possible as well as what they say (in their words). The more you can understand about your contacts’ SMS behavior, the more effective your SMS program will be.

Measure both performance and preference. People’s performance and preferences don’t always match. This is especially true with regard to their mobile phones. Often, users will perform poorly even though their subjective ratings are high. Conversely, they may perform well but give your program a poor subjective rating.

  • Qualitative metrics include: completion rate, time to completion, errors (“other” responses), opt-outs, etc.
  • Subjective metrics include: test contacts’ self-reported satisfaction and comfort ratings.

Evaluating a pilot

At the end of a pilot, you should be able to answer the following questions:

  1. Was the test group’s overall reaction positive or negative? The test group’s feedback can help confirm whether or not your program is a good fit for your population and whether minor changes to the program are appropriate and/or necessary.
  2. Are you allocating your time and resources properly? The pilot will help you determine whether you need to spend more time or resources on particular aspects of your program. For example, you might learn that changes to your method of engagement, flow length, or flow timing are necessary.
  3. Does your evaluation strategy need improvement? Look at this as an opportunity to test your evaluation method as well. Are there metrics you’d like to have that you aren’t collecting? The pilot will give your evaluation and implementation teams a chance to work together before increasing scale to troubleshoot any logistical issues that might arise with the distribution and collection of evaluation data.
  4. Are you ready to increase scale? A pilot can shed light on unforeseen challenges that might arise during a larger-scale implementation, and ensure your team is prepared to handle issues that might accompany an increase in scale. This question is largely dependent on the answers to the others.

Once your service is working as intended, you're ready to test the effectiveness of your flows with an A/B test

Improve the Effectiveness of your Flows with A/B Testing

A vital part of building something is testing it out: build, measure, learn and adjust. At TextIt, we believe there’s no such thing as too much testing. To that end, we provide multiple tools to enable you to test your flows both inside TextIt and out. 

Within TextIt, you can use the 'Split randomly' RuleSet to create equally distributed pathways through which your contacts will randomly pass, enabling you to test the efficacy of your flows. We also provide a handy simulator on the same page you build your flows to allow you to test them as you build. 

Outside the platform, we encourage you to use your 1,000 complimentary credits to measure flow performance and conduct A/B tests pilot users.

A/B testing (also called split or multivariate testing) is a method you can employ to test the effectiveness of two or more versions of a single message, flow or campaign with respect to a desired outcome. It's a type of controlled experiment conducted on similar audiences, the initial version being the control and any additional versions being treatments.

Getting Started

  • Define objectives: What are you trying to achieve with your flow, distribute information, collect data or something else?

  • State your hypotheses: Which version of your flow do you think will perform best? Why?

  • Create a list of the things want to test: For example, flow length, message length, vocabulary and method of engagement (will the contact prompt the flow through a keyword or phone call, or will the flow start on a schedule?).

How to conduct an A/B test

Test one item at a time. 

Let’s say version A addresses your contacts by name while version B doesn’t. When testing two versions of a flow, make sure you’re only measuring the impact of a few changes at a time. If version A is too dissimilar to version B, you’ll have trouble isolating the variable responsible for the result.

Send the flow to similar representative samples. 

Show version A and version B to similar groups and ensure each is representative of the entire target population. If you show version A to only experienced contacts and version B to only new contacts, your results may be skewed. Ideally, you want to split your target population evenly.

Be patient. 

One of the best things about TextIt is time to deployment. You can sign-up, connect a channel and send messages in as little as 5 minutes if you want to, but this doesn’t mean everyone should take that approach. Being that SMS is just as much an extension of an organization's brand as email, larger deployments need to conduct tests to make sure flows are well-received and driving results.

Creating flow versions

Creating a new version of a flow is simple: click the gear icon menu in the flow editor and select the “copy” option. This will produce an exact copy of the flow for you to edit and rename. 

You can link your versions using a 'Split randomly' RuleSet.

Evaluating the results of an A/B test

Completion Rate

In addition to the primary marker of success you select when you identify your objectives, we provide a completion rate for each flow represented by a percent value. The completion rate allows you to see how many of the contacts that have entered the flow have finished it. Comparing the completion rates of two versions of a flow is easiest way to measure effectiveness.

Opt-Outs

Another way to evaluate the effectiveness of a flow is to track opt-outs. Learn more about tracking and managing opt-outs here.

Questions? Comments? Let us know! We love hearing from you. 

Collecting Multiple Answers from a Single Response

Some use cases might call for strict categorical or numeric response guidelines while others might benefit from allowing the respondent to submit multiple answers in a single response, i.e. "indicate all that apply." Here's how you can collect and categorize such responses containing multiple answers: 

In addition to collecting a single response to a question with multiple choices, you can use the "has a number equal to" response rule to allow your contacts to submit multiple choices with a single response. 

Choosing the Right Response Rule

Response rules are directives that categorize or group a contact's responses to an SMS or voice message. The "has a number equal to" response rule will allow you to place multi-answer responses in a single category.

Format your Question

The "Send an SMS response" action below asks the recipient to choose their favorite color, or select multiple if they have more than one. Each choice is coded by a number, allowing you to collect multiple selections in numerical order. This will prevent the need to account for all possible number combinations.



The corresponding RuleSet could then use the "has a number equal to" response rule to categorize each possible combination of responses: 



Any response that doesn't match the response rules above will be categorized as "other." You can use a redirect loop connecting the "other" category to a message that restates the acceptable response format, then gives the recipient the opportunity to respond correctly. 


A few other things you can do with response rules:
  • Categorize similar responses into one category.  For example, if you are asking contacts what their favorite type of soda is, you can create rules that allow "Coke" and "Pepsi" to be categorized as "Cola."
  • Convert numeric responses to narrative responses, or vice versa.  For example, if you ask your contacts to rate their satisfaction with a service on a scale of 1 to 5, you can create a rule that allows the response "1" to be categorized as "Poor," and "5" to be categorized as "Excellent." 
  • Define an acceptable range for a numerical answer. 
  • Direct unexpected responses to an "Other" category. For example, if you are asking your contacts for their gender and receive the response "Cat," this answer is categorized as "Other."
  • Translate responses so that affirmative answers such "Yego" (Kinyarwanda) or "Si" (Spanish) are categorized as "Yes."

Message Content Best Practices

We understand it can be a challenge deciding exactly what to write in your messages. After all, message content can vary entirely based on the population you're attempting to serve. We've put together a short guide to help that process along: 

Message Length

Messages exceeding 160 characters will be sent as two messages. Whenever possible, send one message instead of two. Educating your contacts on your flow's intended purpose and how it works prior to starting the flow lessens the amount of content you'll need to fit into each message. This can be accomplished via demonstrations, training sessions, and marketing efforts. 

Clarity

Be clear with your contacts regarding the types of responses you're looking for and what they can expect from your flow. We advise including all possible responses in each question whenever possible. Use response rules to define acceptable answers and minimize the possibility of receiving responses in an undesired format. The more precise the answers you receive, the more accessible and better organized your data will be. To that end, it's best to categorize responses whenever possible.

In the example pictured below, we ask our contacts to tell us how they get their water. We've used multiple choice response rules to define the acceptable range of answers. These answers will be stored as values in the "Water Source" flow field.



The response rule "has all of the words" indicates any answers that don't have only the designated words "Well," "Tap," "Stream," and "Water Point" will be categorized as "other." 

Redirect

To redirect incorrect responses, you may create a connection from the "other" category to an action step that reiterates the correct response format and then create a connection back to the initial split step.


Language

Make sure your content is appropriate to your target location. This can be accomplished by: 
  • Designing messages that are appropriate for all ages.
  • Using local terminology.
  • Using popular local abbreviations.
While engaging Liberian youth, UNICEF translated the following questions using local, abbreviated language:

“Are you aware of the Ebola disease” became “do pple no abt Ebola.” 

“What information source do you trust” became “which 1 will pple take on d info abt Ebola.”
 
“What has changed the most in your community because of Ebola” became “wat bother U d  most abt Ebola.” 

Managing Opt-Outs through Twilio Channels

Opt-outs are common among SMS ("Robotext") and IVR ("Robocall") services. According to the TCPA (see our guide here), your contacts should have the ability to do so at any time. Use this guide to learn how to track and manage these contacts via Twilio and your TextIt account. 

How Twilio Manages Opt-outs

When a contact sends STOP, STOPALL, UNSUBSCRIBE, CANCEL, END or QUIT to one of your Twilio numbers, Twilio will prevent them from receiving any additional messages until that contact responds START. Specifically, they'll receive the following message:

"You have successfully been unsubscribed. You will not receive any more messages from this number. Reply START to resubscribe." 

When that contact resubscribes, they'll receive the following message: 

"You have successfully been resubscribed to messages from this number. Reply HELP for help. Reply STOP to unsubscribe. Msg&Data Rates May Apply." 

How TextIt Manages Opt-outs

When Twilio notifies us that a contact has blacklisted one of your Twilio numbers, we place that contact in the default 'Stopped' group in your account's Contacts tab. The contact is removed from all other groups and specially marked so that any outbound message attempts are blocked.

(Use this guide to have Twilio notify you of errors–like the 21610 blacklist error mentioned in this article–via email.)

For more information on proper SMS or IVR-based communication, consult our TCPA compliance guide