How to confidently build your research online

2022 Workshop delivered by Sofia Pelica

• Contents

Introduction

PsychoPy can have a sub-millisecond reaction time precision (Bridges et al., 2020), and it's the ideal tool for creating experimental tasks without writing any code!

Check out my Workshop Session with a step-by-step guide on creating a Stroop Task from scratch.

You can also see my pieces of advice and demos!

• Measure the total time

• Hide mouse cursor

• Block mobile devices

• Pipeline with Qualtrics

• Stroop Task with Words and Images

• N-Back with letters and feedback

• Self-paced reading with moving window

• Demos of other users

• Demos in Pavlovia search engine

Feel free to contact me if you have any questions!

Note: Meanwhile, there's a new version. The steps we took in the workshop also apply perfectly to this new version.

Files used during the hands-on: 📎 workshopPsychoPy_files_sent.zip

1. Getting Started

1.1. Installing PsychoPy

1.2. Creating a folder for your project

1. First, let's create a new folder for our experiment. This folder needs to be in the Home folder.
   • macOS users: open any folder. Then, select Go in the upper screen bar, and then Home.

     

   • Windows users: Open any folder. On the left bar, click This PC, then Disk C:, then Users. Click on the folder with your username associated with your computer. This is your Home folder.

     

2. Name it my_stroop. Do not use spaces, symbols, or accent marks on the folder and psychopy file names.

   ⚠️

   Every file in the experiment folder will be uploaded to Pavlovia, so be sure you only put the required files and nothing else.

3. Unzip and copy the files we sent you to the experiment folder.

4. Make an alias on your desktop for easy access.

Save our PsychoPy file:

1. On PsychoPy builder, click on the Save button that looks like an old floppy disk.

2. Choose a name without spaces for your psychopy file (e.g., my_stroop).

   ⚠️

   Do not use spaces, symbols, or accent marks on the folder and psychopy file names. Do not put it on your desktop. Do not put it inside a cloud folder. There could be problems if it's inside multiple folders. Once submitted online, do not change its place.

3. Placed it in the experiment folder that we created inside the Home folder.

1.3. Creating an account on Pavlovia

2. Let's create our experimental task!

2.1. Configure the settings of your study

Click on the Settings button.

2.1.1. Basic settings

1. Choose your PsychoPy version.

2. Confirm that the Escape key and info dialog are checked.

   ⚠️

   You should ALWAYS have the Escape key and Info dialog checked for online studies. The Escape keys allow Pavlovia to send participants incomplete data quickly if they want to leave. The info dialog will enable Pavlovia to download the resources to the participant's browser before your study begins.

3. Add a question about age and gender in the Dialog window:

   3.1. Click + button.

   3.2. Write “age” and “gender” in the left column.

   ⚠️

   PsychoPy online gives an error if you use curve parenthesis () in any cell of the info dialog.

   3.3 In the age question, we want an open response, so we should empty the right cell.

   3.4. In the gender question, we should write some options:['female','male','other']

   ⚠️

   PsychoPy needs to know what words and variables are, so we have to use single straight quotes to let PsychoPy understand that this is not a variable. And the parenthesis needs to be brackets to let PsychoPy know that this is a drop-down list.

   3.5. We want to force participants to answer age and gender, so add an * asterisk. Note that the OK button is unavailable until the age and gender are answered.

2.1.2. Screen settings

For online studies:

1. The full-screen window should be checked.

2. We must define Units. I recommend using height or pix. Be aware that this impacts how we will determine the font size in your study. Let's choose height.

3. We must define the color (i.e., the background color of your study):
   1. Grey background $[0,0,0]
   2. White background $[1,1,1]
   3. Black background $[-1,-1,-1]
   4. Always use this [] bracket parentheses.

• About color space: I recommend RGB or HEX online, but we can write "green" or "white" while programming our task. Be aware that this impacts how we define the font and background color.

2.1.3. Audio settings

This is for offline studies. For online studies, you have to follow the same steps as images (see Adding a Stroop task with images).

2.1.4. Online settings

1. Make sure the Export HTML is selected with “on Sync”.

2. I recommend leaving the other fields empty in this section unless you have a particular objective.

2.1.5. Data settings

Do not change anything in the data filename unless for particular reasons. This defines where data will be saved on your computer and the file's names. By default, it has the name of your study and the date (e.g., ANONYMOUS_my_stroop_2022-02-01_14h26.01.433.csv).

The options for saving are defined in Pavlovia, not here.

Do not change anything in here. Click OK.

2.2. Program the Stroop Task

The Stroop task demonstrates that it is difficult to name the ink color if there is a mismatch between the ink color and the written word (e.g., "blue" written in red ink). Therefore, we will consider the Stroop effect as the average response time in conflict trials minus congruent trials (Oh et al., 2012).

We will only use three colors and two conditions to simplify this workshop.

Congruent condition: The color of the word and the meaning is the same

Conflict condition: The ink of the word and the meaning is different (e.g., “blue” written in red)

Each stimulus will be presented for 3 seconds or until the participant responds.

Participants need to respond to the ink of the words (not the meaning) by pressing a key.

The next stimulus will be presented 2 seconds following the last stimulus when participants respond.

2.2.1. Thinking about conditions

Open the condition_words excel file we sent you.

The first row is the name of the variables.

The A column is the word. We named it “word”

The B column is the ink.

The C column is the correct key that participants should press.

The D column is the Stroop color match. 1 = congruent, 0 = conflict. We are not going to use it as a variable. This is just going to show up in the results file to facilitate our analysis. Instead of 1 and 0, we could write anything here.

2.2.2. Add a Routine

Right-click over the routine Trial. Rename it to stroop.

2.2.3. Add a text component (Fixation Cross)

1. Rename it fixation.

2. Start time: 0

3. Duration: 1.0

4. Text: +

5. Constant

The position (0,0) determines that the fixation cross will appear in the center of the screen.

If we want to place it more to the left, maybe write (-5, 0).

Let's leave it like that.

Opacity is transparency. Contrast is to invert the color (-1).

We want Arial as the font. By the way, we can use ANY google font.

This doesn't work online. If you want to register when a word appears, you need to add a little code.

2.2.4. Add a text component with variable (Words)

1. Name it target.

2. Start time: 2.0 s (because 1.0 seconds for the fixation cross and 1.0 for the empty screen)

3. Duration time: 3.0 s. We should use time limits on online studies.

4. Text: $words. Since this is a variable, we have to put the $.

5. Set every repeat.

6. Write $lettercolour to connect to our variable. This will be determined by the excel file. Variables names are case sensitive.

7. Set every repeat.

2.2.5. Add keyboard response

1. Rename it word_resp.

2. Start time: 2.0 seconds (appear at the same time as the words)

3. Duration time: 3.0 seconds

4. Force end Routine checked. This means this routine will end earlier when the participant presses one of these keys. If we didn't put a duration and check the force end routine option, the routine only ends when the participant presses the key. If the participant goes to the bathroom mid-study, this will be stopped forever. That's why some people advise using the duration limit in everything for online studies.

5. Allowed keys: 'left','right','down'. Note that key names (even for number keys) should be given in single straight quotes, separated by commas, e.g., 'm','z','1','2'. If this box is left blank, any pressed key will end the routine. The space bar is 'space'. Don't use Escape for this.

Store: This asks which key press, if any, should be stored; the first to be pressed, the last to be pressed, or all that has been pressed.

Discard: Check this box to ensure that only key presses that occur during this keyboard component period are used. If this box is not checked, a keyboard press that has happened before the start of the component period will be interpreted as the first keyboard press. This can be confusing to have in your results file.

2.2.6. Add the Conditions file

1. Insert Loop, go around the routine.

2. Choose random.
   • Random - present abc in a random order, because we have nReps at 2, this would be repeated twice (e.g. [c, a, b, a, c, b])
   • Full Random - present abc but it would be possible to get [a, a, b, c, c, b]. The same trial twice in a row.
   • Sequential - present the rows in the order they are set in the excel file.

3. Trial checked.

   Some loops are not trials but a set of blocks. We don't want the data file to add a line in the results file if it's a block, so we have to uncheck this. But this is not our case.

4. nReps: 1

   The number of each row of our condition file will be repeated. We have nine trials, and we don't want repetition, so we will write 1.

   If we wanted a repetition, we could write 2, and then we would have 18 trials.

2.3. Submitting to Pavlovia

1. Click on the second web icon.

2. This is the first time submitting this project. A pop-up asking to create a project will appear.

   

3. Name your official project. This name will appear in the link that you are going to share with the participants.

4. Since this is the first time, we have to go to the Pavlovia site https://pavlovia.org/dashboard?tab=0 and activate our project to pilot mode.

5. Go to Dashboard > Experiment > click on the project. Then click on Pilot under the status. To run the experiment, click on the Pilot button next to the View code.

   

2.3.1. How to open a CSV file

1. Since we are in pilot mode, your browser should download your data automatically.

2. I am going to show you how to open this kind of file .csv on excel.
   1. Open excel.
   2. Go to the Data tab.
   3. Select Get data > From text

      

   4. Select the downloaded file from Pavlovia.
   5. Select Delimited > start import at row 1 > file origin Unicode UTF-8

      

   6. Select comma.

      

   7. Click General.

      

2.3.2. How to interpret the data file

Now you can see what data you have:

2.4. Adding consent, instructions, and practice to our study

2.4.1. Practice with feedback

1. Select Stroop trial. Go to the upper bar and select experiment > copy routine. Go again and select paste routine. Name it practice.

2. Add a new routine. Name it feedback.
   1. Add a text component. Name it feedback_text
   2. Start time: 0
   3. Duration: 0.5
   4. Text: $f_msg set every repeat
   5. Appearance: $f_color set every repeat
   6. Formatting: Arial, 0.05 font size

3. Add code
   1. Begin routine
   2. JS
   3. Add this code:

   if (NAMEKEYBOARD.corr) {
       f_msg = "Correct";
       f_color = "green";
   } else {
       f_msg = "Wrong";
       f_color = "red";
   }

   Go to the practice routine and copy the name of the keyboard component. Paste it to the feedback code, and replace it with “namekeyboard”.

4. Insert loop. Add excel file from practice.

   

   

5. Select practice routine. Update variables' names!!

   

2.4.2. Instruction for Practice

2.4.4. Instructions for the Main task

2.4.5. Thank you message

2.4.6. Consent

2.4.7. Adding a Stroop task with images

1. Copy and paste the Stroop trial.

2. Add an image component
   1. Start time: 2.0
   2. Duration: 3.0
   3. Image: $imageRand set every repeat
   4. Layout: Size (0.6, 0.41). This is the size of our pictures. You should see the size on software like Photoshop or GIMP.

      

3. Change the keyboard to $corrAns_img

4. Go to the Excel file and change the path if you are a windows user because windows use \ and mac uses /. So windows users should change pic/flower_red.jpg to pic\flower_red.jpg

5. Change the loop to img_trials. Insert the excel for this task with images.

   

4. Copy past instructions from practice. Name it instruct_img
   1. Change the text component to instruct_text_img
   2. Change text:

   Now you will see a series of color-filtered pictures, and you should indicate the color of each image as quickly as possible while ignoring the content of each picture.
   
   Remember, ignore the content itself. Press:
   Left for red COLOR
   Down for green COLOR
   Right for blue COLOR
   
   Press any key to continue.

   c. Change the name of the keyboard component to instruct_keys_img

3. Remarks on Data Collection

3.1. Options for saving data

If you choose to save only complete data...

• You will be charged only for completed participants.

• So, if the participant's browser is abruptly closed during the yellow pop-up (e.g., their computer turns off due to low battery), it's considered incomplete data. A log file will appear, but no data.

If you choose to save incomplete data...

• You will be charged for every participant who clicks on Start, even if they close the browser right after that.

• And Pavlovia tries to save the data when the participant double-clicks on Escape. A pop-up will appear to give Pavlovia time to send the data to the server, and the browser only closes when they click “leave” on that pop-up.

3.2. Data format

If you choose database format...

• All participants will be merged into one file.

• And they are ordered by the date of participation.

If you choose CSV format...

• Each participant will have an individual file.

• You can get columns arranged in a different order in each file.

• And the number of decimals used differs! You can get a file with only four decimals on the reaction time. That's less compared to the database format.

So my advice is to choose the database format. If you do so, when you click on Download Results in Pavlovia, you will get a folder with log files and just one CSV file.

3.4. How credits work

When you buy credits, you have to assign them!

• So after buying your credits, click RUNNING. You are going to see a green bar with all your credits. Now assign them, and you will see the bar changing to blue. That is what we want. If you leave it green, the credits are doing nothing to you. They are just sitting there.

• Now you can use your official URL. Don't confuse it with the pilot link you were using.

What are reserved credits?

• Credits are reserved when somebody starts your study.

• It will be consumed when the results are recorded (task completed or browser closed).

• It will be released from its “reserved” status after 24 hours. So if a participant is doing your study but goes to the bathroom and forgets that they were doing your study, you will have to wait 24 hours or until the participant closes the browser or complete your task.

4. Last remarks

4.1. Some notes on troubleshooting errors

• You should google the error message with “psychopy”.

• PsychoPy has a forum for discussing errors. If you need help, go there. But you cannot delete your question afterward.

• Check your component, trial, or file names -> PsychoPy doesn't work with spaces, symbols, or too-long names. Don't use default names either (e.g., “trial” for trial, “text” for text component).

• Check if your experiment folder is in your home folder.

• Delete the js and index file [on your computer] and rerun it to force an update.

• Clean your cookies on Pavlovia to force an update.

• If PsychoPy doesn't acknowledge a new file condition after doing the two previous steps, you should create a new folder and copy your stimuli, condition files, and psyexp file. Associate this with a new project.

4.2. Good practices

• Don't change your PsychoPy version while collecting data or building your experiment! Find previous versions here (https://github.com/psychopy/psychopy/releases).

• Write something on every commit. If your study goes wrong, you can find a working version and restore it. Pavlovia and GitLab save EVERY version of your study every time you commit.

• Don't edit files on GitLab.

• You can fork projects instead of downloading them, so you can acknowledge that person's efforts.

• Configure SSH Key on your Gitlab account to protect the connection between your computer and Pavlovia's server.

• Run pilot tests in different browsers, different computers, and different operating systems.

5. Securing your computer to Gitlab (optional, but strongly advised)

5.1. macOS users

ssh-keygen -t ed25519 -C "email@example.com"

5.2. Windows users

ssh-keygen -t ed25519 -C "email@example.com"

6. Useful resources

• Demos of online studies

• PsychoPy Javascript crib sheet

• PsychoPy Youtube Channel

• Building Experiments in PsychoPy (Book)

7. Extras

7.1. Measure the participant's total time

1. Go to the very last routine you have in your flow.

2. Add a Code component. Change the default name.

3. Change code type to JS.

4. Go to End Experiment

5. Add this text:

psychoJS.experiment.addData("globalClockTime", globalClock.getTime());
psychoJS.experiment.nextEntry();

Note: The last column of the data file will have the global clock time in seconds.

7.2. Hide mouse cursor

1. Go to the routine that has the task where you want to hide the mouse.

2. Add a Code component. Place it at the top of the components. Change the default name.

3. Change code type to JS.

4. Go to Begin routine

5. Add this text:

document.body.style.cursor='none';

Note: If you have another task after, you need to add the following code if you want to un-hide it (following the same steps before):

document.body.style.cursor='auto';

You should enable the mouse again in the last routine, so participants can click “OK” in the green pop-up after completion.

7.3. Stop your task from running if the participant is using mobile phones

1. Add a new routine at the very beginning of your flow.

2. Add a code component. Change the default name.

3. Change the Code Type to JS.

4. Go to Before Experiment.

5. Add text:

if( /Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent) ) {
quitPsychoJS('Mobile device detected. Goodbye!', false)
}

Note: Don't add anything else to this routine.

7.4. Setting up a pipeline with Qualtrics

Imagine you want to insert a large set of questions before and after the experimental task. In that case, I advise you to use a pipeline connecting a pre-survey Qualtrics, then redirect to Pavlovia and send the ID, and finally redirect to the post-survey Qualtrics. You will use two qualtric surveys, but the participant won't notice it.

1. Go to your Qualtrics survey, then to the survey flow.

2. Add two new embedded data fields and name it: participant and session

4. For the session, let's just write qualtrics to make sure what is the origin of the participants on Pavlovia.

5. In the right box, click on insert Pipped Text>Random Number>Integer>min:1 max:1000. Define this as appropriate for your study.

   

5. Go to your study on Pavlovia, and click on Running Mode.

   

6. Copy the link under the Recruitment tab.

   

7. Go back to Qualtrics and edit the end of the survey.

8. Select Redirect to URL. Paste the link of your study hosted on Pavlovia and add:

   LINK?participant=${e://Field/participant}&session=${e://Field/session}

   Example:

   https://run.pavlovia.org/sofiapelica/qualtrics_connection/?participant=${e://Field/participant}&session=${e://Field/session}

   

   This will now pass the Participant ID and session from Qualtrics to Pavlovia.

   You should also add a "Then Branch" and a customized "End of Survey" if participants do not accept the consent form to ensure they will not be redirected to Pavlovia.

9. In Psychopy, go to Settings. Make sure you have participant and session variables in the info dialog and that their right columns are empty. These variables are case-sensitive!

   

10. Then go to the Online tab.

11. In the Complete URL you should add the link to redirect participants after completing the pavlovia task (for example, to the post-survey qualtrics):

    $'LINK?participant=' + expInfo['participant'] + "&session=" + expInfo['session']

    Example:

    $'https://isctecis.co1.qualtrics.com/jfe/form/SV_8c3tgcL5Q6b8pvg?participant=' + expInfo['participant'] + "&session=" + expInfo['session']

Note: In this pipeline, participants start with a qualtrics survey, are redirected to pavlovia, and then are redirected back to qualtrics. A separated post-survey ensures they will encounter the exact items we want and avoid problems with the browser cache. In the end, you will have three files of the same participant.

If you think you only need a pre-survey and the task, skip the last post-survey step.

Pre-survey:

Pavlovia:

Post-survey:

Note: These files resulted from my testing; it's not a participant's data.

Download the PsychoPy and Qualtrics demos:

7.5. Pipeline Prolific → Qualtrics → Pavlovia → Qualtrics → Pavlovia

If you are interested in using these steps to connect to Prolific, you just have to add “PROLIFIC_PID”, “STUDY_ID”, and “SESSION_ID” following the same steps.

Since Prolific will give these variables values, the pre-survey should have "value will be set from panel or URL".

Don't forget to also have these new variables in the info dialog of your PsychoPy study. In the post-survey, the end message should redirect to the Prolific with the code parameters (e.g., https://app.prolific.co/submissions/complete?cc=86319F9F).

If you are not doing the post-survey, then in the Complete URL of PsychoPy you could just put the prolific link directly.

Info Dialog:

Redirect URL from pre-survey in End message:

https://run.pavlovia.org/sofiapelica/qualtrics_connection/?participant=${e://Field/participant}&session=${e://Field/session}&PROLIFIC_ID=${e://Field/PROLIFIC_ID}&STUDY_ID=${e://Field/STUDY_ID}&SESSION_ID=${e://Field/SESSION_ID}

Complete URL:

$'https://isctecis.co1.qualtrics.com/jfe/form/SV_8c3tgcL5Q6b8pvg?participant=' + expInfo['participant'] + "&session=" + expInfo['session'] + "&PROLIFIC_ID=" + expInfo['PROLIFIC_ID'] + "&STUDY_ID=" + expInfo['STUDY_ID']+ "&SESSION_ID=" + expInfo['SESSION_ID']

8. My demos

8.1. Stroop Task with words and images

Files

https://gitlab.pavlovia.org/sofiapelica/stroop-with-words-and-images

You can fork this project or download it.

References

Oh, J., Han, M., Peterson, B. S., & Jeong, J. (2012). Spontaneous eyeblinks are correlated with responses during the Stroop task. PloS one, 7(4), e34871. https://doi.org/10.1371/journal.pone.0034871

Shibasaki, M., Isomura, T., & Masataka, N. (2014). Viewing images of snakes accelerates making judgments of their color in humans: Red snake effect as an instance of 'emotional Stroop facilitation'. Royal Society Open Science, 1(3), 140066. https://doi.org/10.1098/rsos.140066

Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651

8.2. N-Back letters with feedback

This is a 2-back task with letters and feedback. It has 52 trials. The first two trials are considered null.

Each stimulus is presented for 500 milliseconds. People get 3 seconds to respond. A new stimulus is presented every 3000 milliseconds.

When the participant correctly presses the z key because the letter matched the letter two trials ago, it is considered a match (1). When the participant presses the z key wrongly, it's regarded as a false alarm (0). Also, the missed target is coded as 0.

Files

https://gitlab.pavlovia.org/sofiapelica/nback

References

Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55(4), 352-358. https://doi.org/10.1037/h0043688

Jaeggi, S. M., Buschkuehl, M., Perrig, W. J., & Meier, B. (2010). The concurrent validity of the N-back task as a working memory measure. Memory, 18(4), 394–412. https://doi.org/10.1080/09658211003702171

Kane, M. J., Conway, A., Miura, T. K., & Colflesh, G. (2007). Working memory, attention control, and the N-back task: A question of construct validity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(3), 615–622. https://doi.org/10.1037/0278-7393.33.3.615

8.4. Moving Window Self-Paced Reading

This is a moving-window self-paced reading paradigm introduced by Just et al. (1982).

This paradigm allows readers to use button presses to control the exposure duration for each word of a text they read. The latencies of the button presses depend on the properties of the words being read and correlate with the time course of the cognitive processes during reading and text comprehension.

Files

https://gitlab.pavlovia.org/sofiapelica/self-paced-reading-moving-window

References

Just, M. A., Carpenter, P. A., & Woolley, J. D. (1982). Paradigms and processes in reading comprehension. Journal of Experimental Psychology: General, 111(2), 228-238.

Keller, F., Gunasekharan, S., Mayo, N., & Corley, M. (2009). Timing accuracy of Web experiments: A case study using the WebExp software package. Behavior Research Methods, 41(1), 1-12.

Feel free to copy these demos and adapt them to your study. Researchers should support each other.

9. Troubleshooting

Do you have any questions or remarks?

Feel free to contact me! 💬