GUI vs CLI:KLM speed analysis

Introduction

I have decided to analyze what is faster in terms of actual speed of use, the GUI or CLI? In this post I will describe my process, and I will display the results of the various scenarios. All the scenarios describe on how a user would update the software on his computer.

Important:In this test I did not measure the speed of the actual software. What I measured was the total time of how many seconds did the user have to interact(user actions) with his computer to complete the desired goal while using the GUI or the CLI. In other words I am not measuring the speed of the software but the speed of the usage of said software. That is we are measuring the speed of the hand movements and similar(which can cause the carpal tunnel syndrome).

The method

I have used the KLM(Keystroke-level model), it is a method that tries to predict on how long will it take a user to accomplish a routine task while using an interactive computer system. This method assumes that the user knows how to do the taks in question, and it also assumes that in the measured scenario no errors were made by the user.
Keystroke-level model 

I also should mention that although I have used Ubuntu 15.04(Unity) as my reference operating system, I consider that my results are usable on most modern operating systems, because the process of updating the software is very similar with 2-3 common steps:
1)Search for updates
2)User authentication
3)Install the updates

The goal of my measurement was to be system agnostic, so I have simply assumed that the system response time = 0.
Some scenarios however cleverly automate 1), which is perfectly ok for the purposes of this test.
As for the KLM, I have used 4 of its six operators in my scenarios:

• K - Keystroke or mouse button press-I have assumed that the user is an average skilled typist - 1K = 0.2s
• P - Pointing to a target with a mouse - 1P = 1.1s
• H - Switching your second hand from the keyboard to the mouse and vice versa - 1H = 0.4s

• M - The mental preparation for the execution of physical actions - 1M = 1.35s

I have ignored the D operator because I did not use them in the scenario and I have ignored the R operator because this measurement was system agnostic so the system response time was just assumed as 0, as that is not what I intended to measure. The length in seconds for each operation was scientifically established by monitoring many users of different skill levels.

Each of the operators(K, P, H, M) represents one single user action.
In total I have measured 5 scenarios. Each of the 5 scenarios was first broken down to several steps which were then separately measured by the KLM method. When analyzing the steps I will not describe every single user action, instead I will just write down the neccesary operators and the resulting time.

In the GUI scenarios the user starts with an empty desktop with no windows open. The user in those scenarios is using both the keyboard and mouse. In the CLI scenarios the user starts with an empty terminal. In those scenarios on the other hand the user only uses the keyboard. In all of the scenarios the user starts with both his hands on the keyboard. At the begining of each scenario there is one 'M' operation.
The scenarios are:

1. GUI - The user has to find the software update dialogue. He also has to check for the updates,authenticate and install them.
2. CLI - The user has to manually search for updates, authenticate and update his system via the CLI. 
3. GUI - The user gets a 'notification' which leads him to the software update dialogue. The user has to click on the icon, authenticate and install the updates.
4. CLI - The user has to update his system via a script file. He has to start the script, authenticate and approve the updates.
5. CLI + i3wm GUI - the peculiarity of the i3wm GUI is that it promotes keyboard only usage. This scenario acually uses the CLI and the same script from scenario 4. The i3wm GUI however is only used for launching the CLI terminal with the script preloaded onto the terminal. Everything else is pure CLI.

The actual measurements

1)The 'long' GUI scenario

In this scenario the user has to first do 'menu hunting'  until he finds the neccessary system update dialogue box. Then he has to authenticate and search for the updates. After that the user ends his actions by pressing the install updates button.

Here are the steps that I have used to separate everything into smaller chunks:

1.  Finding and activating the 'start' menu: 2M+H+P+K=4.40s
2. Searching for the 'software update dialogue'(typing in 'update' into the search box): H+M+6K=2.95s
3. Activating the 'update software dialogue'(running it): H+M+P+K=3.05s
4. Searching for the updates: M+P+K=2.65s  
5. Authenticating(8 letter password + shift + enter):  P+11K+H+M=5.05s
6.  Installing the updates: H+M+P+K=3.05s

Total: 21.15s 

2)The 'manual' CLI scenario

In this scenario the user has to completely manually input the neccessary CLI commands in order to update his software. The commands used were 'sudo apt-get update' and 'sudo apt-get upgrade'

Here are the steps:

1. Search for the updates: 2M+20K=6.70s
2. Authenticate(8 letter password + shift + enter): M+10K=3.35s
3. Starting the install process: M+21K=5.55s
4. Review and allow the updates: M+K=1.55s

Total: 17.15s

3)The 'notification' GUI scenario

In this scenario the user gets a notification on his GUI that allows him to directly skip into the 'software update dialogue'. Also the 'search' for updates step can also be skipped as it was done automatically by the system.

1. Pressing on the notification: 2M+H+P+K=4.4s
2. Starting the install process: M+P+K=2.65s
3. Authenticating(8 letter password + shift + enter): P+H+11K=5.05s

Total: 12.1s

4)The 'scripted' CLI scenario

In this scenario the user has to start a script that automates the processof searching for the updates and starts the process of installing the updates. It is assumed that the script itself is finished and ready to use.

1. Start the script( './update.sh' + enter): 2M+13K=5.3s
2. Authenticate(8 letter password + shift + enter): M+10K=3.35s
3. Review and allow the updates: M+K=1.55s

Total: 10.2s

5)The 'combined' i3wm + CLI scenario

The i3wm GUI promotes keyboard only usage. It is something that is called a 'Tiling window manager'(Tiling WM ). I3wm functions in such a way that often you can think of it as a visual extension to your CLI. It is definitely not a traditional GUI. This scenario acually uses the CLI a lot more and it uses the same script from scenario 4. The i3wm GUI however is only used for its keyboard shortcuts to launch the CLI terminal with the script preloaded onto the terminal. Everything else is pure CLI. In this scenario the user has to press the keyboard shortcut that opens a CLI with the same script that was used in scenario 4 preloaded. After that the user only has to authenticate and review the updates.

1. Input the keyboard shortcuts: 2M+2K=3.1s
2. Authenticate(8 letter password + shift + enter): M+10K=3.35s
3. Review and allow the updates: M+K=1.55s

Total: 8s

What have we learned?

Well we have learned that the above picture depicts the truth. However there are other things that we have learned as well...

Is the CLI faster in usage?

Definitely. But I will admit that it is a few seconds slower than I expected it to be.

What about those nifty 'Tiling WM's?

Those are also awesomely fast, however in the workflow their nature is closer to being extensions to the CLI. Had we avoided to use the CLI, then we would have had to go trough the same 'menu hunting' process as we did with the traditional GUI(Unity) and that would cost us a lot of speed.
Does that mean that the CLI is superior to the GUI?

Absolutely not. It only means it is faster in usage. Plus there are many other scenarios to test. For some situations and users it is better to use the CLI, for others it is better to use the GUI.
Ok, but what are the advantages of the GUI then?

Intuitiveness and interactiveness. The GUI uses our 'intuitive knowledge' to make it easier to use computers. I.e. you can take apiece of paper, grab it and throw it into the trash can. The same way you can take an icon, grab it and throw it into the trash can. The CLI on the other hand is a lot like 'casting spells' you have to learn the commands and tell them to the computer. 'rm -rf' is very similar to 'Avada Kedavra' :P
Oh and another thing... Notice how the scenarios with 3 steps are faster than the scenarios with 4 and 6 steps...

In the future

 Further research should be done:

1)Further research should be done on many other scenarios

2)UX designers should try to decrease the number of steps neccessary to complete scenarios.

3)UX designers should implement more keyboard shortcuts for its software even when it is not marketed only towards experienced users.

4)When designing GUI's for advanced users, UX designers should try to integrate the CLI into the GUI.  A good example would be how the file manager named 'Dolphin' has a CLI extension(picture below):

5)Tiling WM's are nice and they are something that should be looked into.
6)The strength of the GUI is in its intuitivity. However when it comes to doing more with less the CLI has the advantage. That means that the GUI world must not stagnate, in order to start doing more with less UX designers must do 'wild exerimentation'.
7)Where are NUI's in this whole mess?