Introduction
As students of NC State University, we will all use MyPack Portal several times throughout our collegiate career. Of the more than 34,000 undergraduate and graduate students, everyone will use MyPack Portal at least eight times, to enroll in classes for the following semester. Although, more students end up using it at least a few more times than that, whether this be to view financial aid or decide what classes would be best for them to take. Thus far in my collegiate experience, I have had to rely on MyPack Portal many times. Just a few of those reasoning are due to financial aid, my student account, signing up for on-campus living and class enrollment. I was forced to become accustomed to the layout because there was no other way to navigate the site. Although, as the site was updated only a few short days before class enrollment, no one was confident in their use of MyPack Portal. Many people seemed to be getting very upset by the site before even learning how to navigate it. This concept seems to come from the idea that it was something new during an inconvenient time. Students did not want to take the time to have to learn to learn the site when they were primarily focused on enrolling in their classes and being able to prepare themselves for a successful semester to come.
An article by Noah Jabusch states, “Nonetheless, since previously MyPack had all the page links laid out on a single paid, some students may not be used to paying attention to the folder headings.” There have not been any items taken away from MyPack Portal, they have been organized into categories. A person once had to click on the ‘Wolfpack Student’ tab and it would bring up a link to every page that could be found on the site. In a sense, the change that was coming to MyPack Portal could have been expected. The previous site had all the categories that currently exist but everything could be seen at once, whereas; now there has to be an extra click or two to get to the desired location. The old site, however; was not a positive site. It was not incredible due to the fact that there were always people complaining about the site and its layout. A person would think the new changes would be seen in a more positive light by students. So far, this does not seem to be the case.
From my study and further observations, people simply do not like change. Jabusch tells, “Change is often frustrating as we are forced to adapt to the revised organization and relearn how to complete tasks under the new infrastructure.” From what I have found, no one was particularly in favor of the old MyPack Portal, they were just used to its flaws. This is what brings forth the dislike towards the new MyPack Portal, it is different and came at an inconvenient time. I believe that if MyPack Portal had been updated after everyone enrolled for classes or before the school year began, there would not be so much disapproval to the site. People would learn to use it because it was new and they had to, but there would not be as much uproar as there has been.
This test was also necessary to see how the users reacted to the site while on it, not just talking about it. It is necessary to see what improvements could still be made by the student and the site. Overall, there have always been complaints about MyPack Portal, but whether the reasoning behind this is logical or not is what I am discovering during my usability test.
Description
During this test I want to learn how people are able to navigate MyPack Portal and, more specifically, how they are able to find the Enrollment Calendar and the block of time in which they would enroll. I also want to see the reactions that my users have towards the site and the navigation. For example, if they get frustrated or as if they want to give up on the task, or if they are able to easily find the page.
For my users, there will be an age range throughout each grade. I plan to test a freshman, sophomore, junior and senior to see how the level of overall knowledge for the site varies. I was expecting the older students to know exactly how to find the page that is being requested of them and the younger students to have a bit more difficulty. Although, this was much easier to think about in the beginning stages of the usability test because MyPack Portal had not yet been updated. Now that it is completely new to everyone, I plan to see if the knowledge-based use is still prevalent. I am a little weary of what to expect, but now I am more expecting the younger students to be more confident in what there are looking for since they do not have as much previous experience using the old MyPack Portal.
To test the users, I will set up my laptop on the Google homepage and the user will have to navigate to the Enrollment Calendar from there. They will be using my personal account to search for the site to cut back on the time it takes and so they are not revealing any of their personal collegiate information. I will be recording the video on my phone to see their movements on the screen as well as to capture any expressions that are made. I will also be using IOGraph to trace the movement of the mouse, then I will compare all of the results together to see if there is a correlation and evidence that can be found. I will encourage the users to talk out what is happening, this way I will be able to judge more of how they are feeling about the test through the words they say and the tone they speak in. I want plan to have as many ways as possible of information on the participants. With each different type of data collection that is used I will receive different knowledgeable information that will be very useful when I am making conclusions on the test.
Norman Relation
When the users began the test the typical perceived affordances were to click and search until they found the correct location. This began typing into the Google search bar ‘MyPack Portal’. Once the users were on the search page, the desired website was the first option at the top of the list. The user then clicked on that, logged in and were then taken to the main page on MyPack Portal. This is where the “affordances that are easily discoverable, the perceived affordances” (145) ended. All of the users were less confident in their actions when they entered the MyPack Portal. The test began with the perception of what actions needed to take place but quickly deteriorated. This is heavily due to the new change that resulted from MyPack Portal being updated. The users were still not as familiar with the site as they would have preferred.
Once the users were on the main page of MyPack Portal they began to complete rule-based actions, this due to the “wrong plan being formed” (171) by the user due to the lack of knowledge on using the new site. Each user knew the goal that needed to occur, yet they decided upon the wrong rules and actions to follow. The amount of times this occurred varied due to the personal knowledge the individual user had with MyPack Portal.
The signifiers, “the communicators where the action should take place” (14), were the actions that helped the users know best of what do to complete the task that was at hand. This was made more possible to accomplish due to the affordances, both of these are needed for the task to be as successful as possible. The main signifiers were the words that were used to describe each folder. Each user went to the correct folder the first time due to the word “Enrollment” being in the description. This caused a small lightbulb to turn on in the user’s brain. They connected the goal of the test, to find the Enrollment Calendar, with the language used on the page to know where to go from the main page.
The users then had to use the knowledge they had on MyPack Portal and the ‘clues’ from the site to continue searching for the page. The use of mapping and feedback comes into existence during this part of the test. There was a small amount of natural mapping that was used due to the users knowing which folder to use. The users “took advantage of the spatial analogies” (22) to then find the Enrollment Calendar. There was not an immediate understanding due to the site still being vaguely unfamiliar. The designers of MyPack Portal seem to have had this concept in mind, due to the fact that the folders and subfolders with the word “enrollment” were the ones used to find the designated page. The same pattern was kept to make it easier on the user to find the page, no matter how much previous knowledge they have with the new MyPack Portal. Through the feedback, “the communication of results of an action” (23), is used when the users go back and forth between the few similar pages. Each one knows the desired page has to be in that area but they still have trouble locating the page. Due to not being able to easily find the designated page after using the signifiers and affordances, the users began to give more feedback, such as straightening of posture because they were getting frustrated in the layout of MyPack Portal. The more frustrated the users got, the more feedback they gave. Overall, the signifiers and affordances encouraged the user where to go, but, in the end, it took the knowledge of the user to correctly find the Enrollment Calendar.
User Identification
Alexis Baldwin: freshman
Alexis is a freshman at NC State University. She has had previous experience with the MyPack Portal before it was updated. Since she has only been on campus one semester most of her previous experience occurred when she was accepted, and before she actually began taking courses at the university. She said she used it to register for her classes and view her financial status. Since the new MyPack Portal has been released she has had a much harder time using the site. In the sense of her courses, she was able to pretty easily figure out how to use get to the Enrollment Wizard and add courses to her shopping cart. The area she has had the most difficulty with is the finances. She joined a Panhellenic sorority this year and while she is very excited for this experience, she still has to focus on saving money and being financially stable. She chose to do this by comparing her living accommodations for next year. She knew it was once possible to view living on campus and living in her sorority house and wanted to compare the prices. She was very familiar with how to do this on the old MyPack Portal but when she was attempting to do the same on the updated version, she had a large amount of trouble. A week or so after the update, she sat down with her mother and they tried comparing the prices for her possible living arrangements for next year. She says it took them more time to figure out how to find the location where this information was than it did to actually do the research. Since then, she has become more accustomed to the site, at least the areas she needs to know, but does not think it is any better than what it once was.
During her test, Alexis was having a difficult time. She either kept forgetting or did not comprehend what the Enrollment Page actually was. She began my going back and forth between the Enrollment Wizard and semester plan on the main page and the Pack Planner. Around half way into her test, she found the Enrollment Calendar but did not realize it. I believe this is heavily due to that she was taken out of MyPack Portal to another site. She went back to the home page to try and decide if it would be under another folder, but none others seemed to be reasonable. She also questioned if the enrollment time that was listed on the Planning and Enrollment folder was correct, due to this being my personal enrollment time, it was not what the users were asked to find. She went back and forth to the separate page of the Enrollment Calendar and MyPack Portal several times. After a few hints that she had already been on the correct page several times she then went back to the Enrollment Calendar. I did not want my users to just find the calendar and finish the test, so I told them to find when their block would be. Several times Alexis scrolled up and down on the page but did not know that was the correct one. I then gave another hint that she was very close to where she needed to be and she then scrolled more carefully on the Enrollment Calendar page. She was then able to find her block of time for enrollment, November 6-8, after almost five minutes of taking the test.
User Identification
Hannah Wright: sophomore
Hannah is a sophomore at NC State University. She has had more previous experience with the old MyPack Portal than the new so she was a little skeptical about taking the test. When it comes to her experience with MyPack Portal, she does not have much more knowledge about it than the base information. She typically only uses it for course scheduling and occasionally checking her GPA. When it comes to the navigation of the site, she is a little unfamiliar with it simply because the resources she uses can all be found in the same folder on MyPack Portal. She has not had to learn how to use the rest of the site because she has not needed it. She does think that the new design of the site is much better. She says that it is much easier to access important information, such as the Enrollment Wizard. The resources she has had to use she has not had much difficulty navigating to after being accustomed to the previous design of MyPack Portal. Although, she can only use the site on certain web browsers. For instance, she typically uses Google Chrome but is only able to successfully use MyPack Portal on Safari. This is a small conflict that she has been easily able to adapt to. Although, she says that if she was having to find an unfamiliar resource on the site, she would not be as confident in her actions.
During her test, Hannah did not have high hopes about finding the Enrollment Calendar and being able to complete the task of the test. Throughout the test, she was not confident in any of her actions. When she began she went right to the Enrollment Calendar page but because it took her to a different site, she just assumed it was incorrect. She then went through the tabs in Planning and Enrollment folder a little more carefully to see if she could find any context clues as to where the page could be. Once she did not find anything there, she went back to the MyPack Portal homepage to examine more about the folder options to discover if there was another option for where the Enrollment calendar could be. She went to the Academic Records folder but only stayed on there for a brief moment to analyze what the tab options were. After she realized it was not where the calendar was going to be. She went back to the Planning and Enrollment folder, assuming it had to be in that location. She then analyzed the Enrollment Wizard and Advising Dashboard tab more thoroughly then before. She then went back to the Enrollment Resources tab to click on the Enrollment Calendar category. She scrolled through the page this time, whereas before she immediately exited off the page because it took her out of MyPack Portal. She then found her enrollment block, November 1-2, after a little over two minutes of taking the test. She said afterwards, that she was expecting to find a page with every student and then be able to type her name in and find her personal enrollment calendar through that resource.
User Identification
Susan Donaldson: junior
Susan is a junior at NC State University. Most of her use with MyPack Portal has been through registering for classes and checking her bill. She uses the site a couple times a week during financial aid times and class registration. She says she does not use the site much other than that. She has the primary knowledge of how to use the site due to using these main resources with the previous MyPack Portal. At first, she was not very confident in her ability to use the new MyPack Portal because of the way it was laid out. She did not really understand it or know how to learn the update. Although, she says that after she has used it more commonly, such as for registering for classes only a few days ago, she has gotten more of an understanding of the site. She says she actually enjoys the new design because of the toggle options that it offers, which was not previously offered in the old MyPack Portal.
Before her test, Susan was excited to participate in the experiment. I had tried doing the test with her several times but we were locked out of MyPack Portal, most likely due to the amount of times I had been using the site. Because of this, she knew the goal of the test before she was actually able to partake in it. Although, when I was explaining it to her the first time, she was very confused about the goal of the test. This confusing was still evident when she was taking the test, but it was not as strong this time. I had to explain to her more of what the page was like than with any other user. The first question she asked when the test began was if any other people knew where this page was located. This seemed to be due to the idea that she was not as confident in her actions and wanted to know if it was just her usability of the site or if it was a common error. From the homepage she went to the Planning and Enrollment folder. She did not click on any of the tabs because she was still uncertain about her relation to the site. She held the mouse over the Enrollment Resources tab, the correct one, the longest, then went to the Advising Dashboard and back up through the rest of the tabs. She then assumed she was in the completely wrong location for where she needed to be to find the desired page. I had to reassure her that she was in the correct folder and she then went about analyzing the tab options a little more carefully. She went to the Enrollment Resources tab and looked over the options before clicking on anything. Once she saw the Enrollment Calendar, it is as if a light went off in her head as to what steps needed to be taken next. She looked at the options at the top of the page before randomly scrolling to see if there were any more clues. She then was able to find her enrollment block, October 26-30, in a little over a minute.
User Identification
Katie Peters: senior
Katie Peters is a senior at NC State University. She does not have an extreme amount of previous experience with the old MyPack Portal and uses the same features on the updated site. She primarily uses the site for degree audit and to enroll in her classes. She had become very accustomed to the process that was necessary for the previous site and was able to adapt to the steps necessary. She was used to that site and had used it for the majority of her collegiate career. With the new MyPack Portal, she has a little more difficulty doing these tasks. She says she was able to find these resources after a few extra minutes of searching for them than she would normally have to do. She thinks the updated MyPack Portal is very nice, but since she will not have to use it after next semester, she does not feel the need to learn a detailed amount about the site because it will not be necessary in the future.
Before her test, I had asked Katie a couple days in advance if she would be able to partake in this test. I had given her the goal of the test so she could judge the time, since we are interns at the same organization together. I had not done this with any of the other users because I did not want them to have the opportunity to think deeply about the test and what they would need to do to successfully complete the test in a shorter amount of time. Katie was a little nervous about the test, this seemed to be mostly due to the fact that she was a senior and had not really heard about the Enrollment Calendar. When she began the test, she went directly to the Planning and Enrollment folder and then to the Enrollment Wizard tab, where she scrolled throughout my classes for the upcoming semester and assumed that was it. She then got very nervous and did not know what to do so she went to another tab and then just back to the home page. There were no other folders that seemed to fit the identification of where the desired page would be, so she went back to the Planning and Enrollment folder. I had to reiterate to her several times what she was looking for during this part of the test. She scrolled throughout the tabs several times and then decided to click on the Pack Planner tab. She went back to the Enrollment Wizard then to a menu bar at the top of the page before going to the Enrollment Resources tab. After this, she went to the Enrollment Calendar page and scrolled throughout the page before asking, again, what it was that she was looking for. She went back to the top of the page to analyze more of the options on the page, but instead of clicking on one, she scrolled back down to the general dates for when enrollment is. She was still a little confused if she had found the correct page or not, but was able to find her enrollment block, October 23-25, after taking a little over two minutes to complete the test.
Outcomes
The outcomes of the test came out as I had expected, which I was surprised by. I believe the knowledge of the site increases with age and from my results I was able to tell that just because the site was changed does not mean the prior knowledge of the previous layout of MyPack Portal was no longer needed. From the youngest to the oldest user, none of them knew what the Enrollment Calendar was or where it could be found. They each went into the test blind to what was going to be discovered. While the time does not show as well of an accurate pattern to the knowledge-based idea, the mouse tracking does. With each change in grade level, the amount of mouse movement throughout the site decreases. The older users are thinking more intellectually by looking at the site and deciding where to go next, while the younger users are using the mouse as their way of thinking. It is as if, when the mouse moves so does a thought in their brain, they are thinking in the same moment as the mouse is moving.
From my study during this test, the users primarily relied on signifiers to help them accomplish the test. For example, the word “enrollment” appears in each category that has to be clicked on to get to the calendar, from the folder to the resource inside the tab. Once the users figured this out, I believe they were able to be more successful in their efforts to finish the test in timely manner.
MyPack Portal shows the affordances, “the potential acts that are possible” (145), through the use of the folders and categorization. This seems to be one of the positive changes from the previous to current layout of MyPack Portal. There is much more organization in the new setting which reveals to the user how much further they will have to go to reach a certain point. This is seen through the test. The users know immediately when they are brought to the home page of the site that they will have to go further than that to even attempt to find the desired page. Through the use of signifiers and affordances, which are both needed, the users were able to complete the test by finding the Enrollment Calendar.
This test shows how the advancement of MyPack Portal was a positive move. The categories that each items are divided into shows the need for the organization. However; there is also a slight learning curve due to these additions. Being as the change came at an inconvenient time, students were focused on learning only the necessary tools to register for their classes, they did not have time to learn the whole site. This test shows that it is not as difficult to learn the new site as one might expect it to be. It takes the user dedicating a few extra moments to learn the site. There could even be a tutorial or campus-wide email sent that shows the basic steps to find the most commonly used features of MyPack Portal.
Reflection
The test worked very well. While the primary goal was to test the usability of MyPack Portal, there was a lot more information that was learned during the test. I was able to be a first-hand witness at how other students interact with the site. Whenever a person talks about MyPack Portal the people around tend to groan or have a negative outlook on the site. From this test, I learned that it is because those users do not want to learn the site. The longest test time I had was five minutes, which is not a lot of time, considering the user was very new to the site, in general. This test shows that all it takes is a student allotting about ten or fifteen extra minutes, this could be before they are registering for classes or if they are viewing their financial aid, to attempt to learn the site and find the resources that are most beneficial for them. Whenever something new appears in the world there is always time committed to learning how it works and what tools are most necessary for the user to comprehend based on their personal wants and needs. The users I chose for this test were very willing to learn how to find the desired page. It is something beneficial to learn and could end up helping them in the future.
The communication with the user during the test did not work as well as I would have hoped. They were very willing to speak and react to moves they were making on the site, yet when they became confused the most obvious thing to do was ask for help. I struggled with finding a balance between helping them with the step they were on and giving away how to find the page. I found that if I repeated the goal of the test then they would understand what step needed to be done next before I gave away any more information.
There were system errors on MyPack Portal that occurred while I was giving the test that did not help in the data collection or knowledge of the user. The first test I did ended up not even being used because the site would not load at all after ten minutes of the user trying. The user closed out and reopened the site several times and attempted to open the page in different ways for the site to work, yet nothing happened. This shows the flaws of the site not responding as well as it needs to.
If I were to do this test with a larger group, I would take pool of people from each different grade, but there would be more of a variety of people. I would try to get users that were involved in many different aspects on campus to see if that knowledge was able to play a role in helping the user find the desired site. I would also want to record the facial expressions of the users. I was able to see them during the test and then hear the voices and assume what their face looked like in that moment. Although, it would have been helpful to have a video of the expressions to look back on and be able to analyze with more depth.
How to locate the Enrollment Calendar in MyPack Portal
Resources
“ETSS Portal Entry.” ETSS Portal Entry, mypack.ncsu.edu/.
Jabusch, Noah. “OPINION: MyPack Portal Redesign Is Sleek and Effective.” ProQuest, Uloop, Inc., 16 Oct. 2017,
search.proquest.com/docview/1951012809?pq-origsite=summon.
Norman, Don. The Design of Everyday Things. Basic Books, 2013
tracy sewell 