Results of Review 

Definition of Computational Thinking 

Over time, the concept of computational thinking has developed and become more precise. In the book Mindstorms: Children, Computers, and Powerful Ideas, published in 1980, Papert introduced LOGO, the first programming language for children, and its potential benefits in children’s education. Papert claimed, “computational ideas can provide children with new possibilities for learning, thinking, and growing emotionally as well as cognitively” (pp. 17-18). “Computational thinking” was first mentioned in this book with his vision to integrate it into everyday life aspects (p. 182). However, CT did not receive much attention and development in the two decades that followed. Until 2006, Wing published a three-page long editorial to emphasize the importance of CT and gain widespread acceptance. She defined “Computational thinking involves solving problems, designing systems, and understanding human behavior, by drawing on the concepts fundamental to computer science” (p. 33). Compared to the first definition from Papert, Wing’s statement has the same foundation; they both defined CT as a thought process, and it was first developed in the CS field. Wing, on the other hand, provided a detailed explanation of the thought process, which included “problem-solving,” “design systems,” and “understanding human behavior.” Ten years later, Wing published the next research paper and had a more detailed explanation of her first argument that CT is “thinking like a computer scientist” (Wing, 2006, p. 35). She said, “Computational thinking is the thought processes involved in formulating a problem and expressing its solution(s) in such a way that a computer—human or machine—can effectively carry out” (Wing, 2017, p. 8). This definition is both broader and more specific than the previous. Problem-solving and design systems were clearer when put in the sequencing process of “formulating problems,” including finding and forming new problems, then explaining the designed solutions for other people and computers to perform. This concept is also expanded because solutions can be implemented by both humans and machines. It implies that CT does not need to rely on CS and programming to solve the problem. Even without computers, CT can assist individuals in defining and implementing successful solutions for all parts of life; however, this does not negate the importance of computers in the implementation of solutions. 

Since Wing’s first definition of CT in 2006, there have been several iterations of its definition. One of the most prominent is Hemmendinger’s definition in 2010. Hemmendinger disagreed with Wing on the definition of CT as thinking like computer scientists (Wing, 2006). He argued CT “is to teach them how to think like an economist, a physicist, an artist, and to understand how to use computation to solve their problems, to create, and to discover new questions that can fruitfully be explored” (p. 6). Hemmendinger wanted to expand the definition that the ultimate goal of CT should not be to educate everyone on how to think like a computer scientist, but rather to teach people how to use computation skills to solve problems, generate and explore new ideas across all fields. However, when Wing added more detail to her explanation about CT in 2017, it aligned with Hemmendinger’s claim. In Wing’s definition (2017), she also expressed the vastness of CT and did not encapsulate it in CS. When humans can carry out the problem and solution, it is not limited to any field. Thus, Wing’s new definition can cover Hemmendinger’s (2010) definition. In addition, the concept of CT was started in CS, so it is reasonable to insert the computer aspect in the definition. In this review of research, I will define CT based on Wing’s updated definition in 2017. CT is the thought process to form the problems and design effective solutions for that problem. The whole process can be executed by humans with or without support from computers. Based on this definition, I will continue to explain the elements of the CT concept.

Elements of Computational Thinking

Since there is no concrete definition of CT, researchers choose different elements of computational thinking to focus on. In this section, I will compare ten approaches to find out what they have in common. In addition, based on my definition in the previous section, I will highlight the six key elements of CT that encompass all other elements: decomposition, abstraction, data, algorithm, iteration, and generalization. These elements will be present in the thought process including forming problems and providing effective solutions in many aspects, not just in CS.

Figure 3 illustrates the frequency of each element in ten different studies. These ten studies are the top searched and most recent articles which directly focus on CT and CS in the ERIC (Education Resources Information Center) database and Google Scholar. Ten research papers are examined including Anderson, N. D., 2016; Barr & Stephenson, 2011; Bers et al., 2014; International Society for Technology in Education (ISTE) & Computer Science Teachers Association (CSTA), 2011; National Research Council, 2010; Shute, V. J. et al., 2017; Sneider et al., 2014; U.S. College Board, 2022; Wing (2006, 2017); and Yadav et al. 2016. Two elements are mentioned in 9/10 articles are abstraction and algorithm. Wing argued, “abstraction is key” (2017, p. 8), and algorithm appears in all aspects of life (Wing, 2006, 2017). The following essential elements are decomposition (6/10), automation (5/10), generalization (4/10), data (4/10), and iteration (3/10). Elements such as simulation, modeling, programming, and debugging are less frequently cited. 

Figure 3

The Frequency of Computational Thinking Elements in Ten Examined Studies 

There are two reasons for this rarity. First, the rarity elements were covered by broader elements that had already been mentioned. For example, simulation and modeling are the results of the abstraction process. People observe the problem, choose the most important factors of the problem, then generate a simulation or model to illustrate the abstraction of the problem. Similarly, debugging is a part of the iteration process. The solution will be implemented, evaluated, and improved in the interaction process. The second reason is that these elements are too focused on the CS aspect. The CT definition includes the idea that humans or machines can formulate the problem and execute solutions. Thus, CT core elements cannot be strictly connected to CS and programming. It must be general elements that can be applied with or without a computer. Since elements such as programming and the internet are not valid for all situations, they should not be considered core CT elements. 

  Based on the review of current literature, the definition of CT mentioned above, and my knowledge of CS, I chose six core elements of CT that are essential in the processes of forming problems and designing solutions: decomposition, abstraction, data, algorithm, iteration, and generalization. Decomposition divides big issues and requirements into smaller parts to make them become more accessible to solve. Combined with decomposition, abstraction helps remove the detail to formulate the problem easier. For example, when students need to investigate how microwaves heat up food, they can focus on the parts that create the heat instead of paying attention to all the components of the microwave. In other words, the abstraction helps remove unnecessary information and focus on the main problem. Data is essential in facilitating problem-solving and decision-making (ISTE & CTSA, 2011). The data element here includes data collection and data analysis or interpretation. With data evidence, it will be easier to find out the problem as well as validate the solutions. An algorithm is a step-by-step instruction to solve the problem. Algorithms are powerful because they are the guideline that helps humans or machines execute the solution. A good algorithm will be optimized to have the highest efficiency. Iteration is the element that focuses on the loop of the computational thinking process. Defining a problem, designing a solution, and implementing it is not a one-way flow process. All steps should be in the feedback loop in order to debug, improve, and optimize. Generalization extends the problem and the solution to be applied in various situations. Generalization and abstraction are sometimes not distinguished. Abstraction removes unnecessary characteristics of the issue, so we can focus on solving the main things. On the other hand, generalization expands the solution, so it can be applied in similar situations or even the current solution in the broader problem. In conclusion, these six essential elements: decomposition, abstraction, data, algorithm, iteration, and generalization clarify the definition of CT, and are used as a guideline in designing curricula and assessments.

Computational Thinking Practices in the Two Curricula

Based on Wing’s CT definition (2017) and six primary components of CT, including decomposition, abstraction, data, algorithm, iteration, and generalization, this section will examine how the curriculum supports students in developing each CT element. Based on my cross-analysis, both Harvard and STEAM for Vietnam curriculum are fully integrated decomposition, algorithm, iteration, and generalization. However, abstraction and data need to be studied further so that the curricula can teach these elements more effectively. In addition, both curricula need to integrate more connections between CT in CS and how students can apply CT in their daily life.

Decomposition

Decomposition is the skill that helps students to deconstruct a problem into small and manageable parts. Decomposition is frequently used in both curricula. Each unit in the STEAM for Vietnam curriculum starts with introducing a sample project, and then the teacher will guide students to decompose that big project into characters and actions. Repeating this step in all units creates vast opportunities to practice decomposition. In the Harvard curriculum, the method to complete projects for the first three units is starting with small ideas and gradually upgrading them by adding more code, which is not using decomposition. However, from unit four, which is Games to unit six, which is Hackathon, students frequently observe their favorite games, brainstorm ideas, and decompose big ideas into smaller parts that are implementable. Both curricula create many good opportunities for students to practice decomposition by creating different Scratch projects.

Each one of these curricula included the decomposition element in the curriculum, but the skillful level of decomposition is not mentioned. Students who are at different learning progressions may decompose the same problem at various levels. For example, beginning students can only analyze jump actions in Scratch as a combination of steadily going up and down. However, expert students will decompose jumps as decreasing motion up and increasing movement down. In other words, students are more skillful in decomposition when they can decompose complex problems into more detailed tasks. Consequently, curriculum developers should pay more attention to the complexity of decomposition in each unit to provide suitable resources to help students not only learn about decomposition but also can improve their skills.

Algorithm

An algorithm is a list of instruction steps to finish a task (Code.org, 2022). The primary purpose of programming is to write instructions steps for machines to follow. Thus, learning programming is learning to write algorithms. Since both curricula focus on teaching CT through CS, especially programming, the algorithm aspect is highly focused. Indeed, in the first lesson of the Harvard curriculum, students analyze the sequence of dancing steps in real life to create a dancing code in the following lesson. Similarly, in the STEAM for Vietnam lesson, after decomposing actions for each character, teachers and students will write algorithms in the Vietnamese then transferring them into Scratch language.

Algorithm optimization is a missing point in both curricula. There is always more than one way to complete things. Thus, the problem solvers not only need to solve the problem, but they also need to optimize their solutions. However, optimization is a challenging aspect. Whereas both curricula are for beginners, so it’s possible, for this reason, the curriculum developers may have purposefully designed the curricula to account for novices. Algorithm optimization should be considered to add to more advanced courses.

Iteration

Iteration in CT is the repetition of a process that includes trial, error, and improvements. This is an essential characteristic of the software development process. A short feedback loop can be beneficial because it helps students learn by doing, which is a proven effective learning method (Dewey, 1923). If students are trained to not affairs of making mistakes and are determined to try out different plans to investigate and solve the problem, they will be more proactive and thrive in the future. 

Both curricula use Scratch as the main programming language, so it supports students in learning and practicing iteration. When students create a new program and run it, they can see their results display immediately on the right part of their screen. Students can compare the result with what they expect. If those do not match, they can easily change the code then try to run it again. Because the process of trying and testing is simple, students are not afraid of attempting different solutions. Iteration also happens when students perform debugging – finding and fixing bugs in their program. Bugs are unavoidable when programming. During the STEAM for Vietnam lessons, there were many moments when both teachers and students need to slow down the coding progress and fix bugs. The Harvard curriculum also highlights the interaction process by designing a separate lesson to focus solely on debugging in each unit. Thus, both curricula highly support students in developing the iteration element.    

Abstraction

Abstraction in CT is removing unnecessary details to only focus on the core characteristics of the objects. This is CT’s most critical and high-level thought process (Wing, 2017). Abstraction can be shown through defining patterns, generalizing from specific instances, and parameterization (Wing, 2017). Abstraction is included in both Harvard and STEAM for Vietnam curricula, but it is not explicitly explained and does not have various examples as other elements. Particularly, the Harvard curriculum does not mention the abstraction term with students. Abstraction is only implicit teaching through examples. For example, students learn to create their blocks from unit three to modularize code and define patterns. However, in the following lesson, all the code goes directly into one main program, and there is no focus on modularizing code anymore. Another example of abstraction is students using variables to calculate scores for players or using clones to create many duplications from one specific instance. Compared to the Harvard curriculum, the STEAM for Vietnam curriculum explains abstraction more explicitly. Abstraction is one of three analyzing steps repeated in all units before the coding part. STEAM for Vietnam also consistently uses functions to help students learn and practice modularized code. However, similar to the Harvard curriculum, the application of abstraction is still limited, only focusing on using the function, variable and clone. Thus, the future curriculum needs to focus on this element to make it more explicit and give students more chances to practice abstraction through various applications.

Generalization

Sometimes, abstraction and generalization are mixed together because abstraction is a way to generalize. In this literature review, I want to distinguish between generalization and abstraction. Abstraction decreases complexity by hiding unnecessary detail while generalization reduces complexity by replacing various entities that perform similar functions with a single entity. As a result, in computer science, generalization is the process through which computer scientists generalize a solution so that it can be reused for comparable problems in the future.

Scratch programming language has a lot of functions that support students to use generalization to optimize their coding process. Since both programs employ Scratch as the primary medium for teaching programming, they help students develop generalization skills. Some of the functions that support generalization in Scratch are copy code, duplicated sprite, and backpack which is used to store students’ favorite code. Both curricula help students to use these functions effectively. For example, when students implement a function that is similar to a function that students have done in the past, they are encouraged to reuse the code. Similarly, when students program similar objects, they are able to duplicate the sprite and only change the different parts instead of making the whole code from the beginning. Consequently, generalization is widely mentioned in both curricula. By seeing the connection and the similarity among projects, students have many chances to use generalization and take advantage of it for faster coding.

Data

The data element, which includes data collection, analysis, and representation, is an integral part of CS. Data collection and analysis help formulate the problem, and data representation is part of an effective solution. In both Harvard and STEAM for Vietnam curricula, data is only at the introduction level, where students learn about storing and displaying data. For example, students learn about data by using a variable to count time and calculating players’ scores when they program games. Furthermore, in the STEAM for Vietnam curriculum, the data concept also expands to using the list to store multiple data instead of using variables to store single data. However, it only includes collecting data to add to the list then displaying them on the screen. There is a lack of processing and representation of the data to support solutions. Data is an integral part of CT, but it is also a difficult programming aspect. Curriculum writers should work more on this element to give students more chances to work with data, even with unplugged activity, to help students learn about data collection, analysis, and representation. 

Daily connection

Because both curricula use programming to teach CT, they are missing the aspect of expanding using CT skills in daily life. In the Harvard curriculum, the last lesson, which is Hackathon, allows students to create a project based on their own ideas to solve their found problem. It can be counted as a daily connection. However, the final result is software, so it also narrows down the CT application inside CS. Likewise, the STEAM for Vietnam curriculum has one lesson that provides students with a context that they are lost on a deserted island. In that situation, they need to decompose the situation, find out the most important thing to survive then create plans to go home. This is an effort to help students think about CT concepts out of CS; however, it might not be realistic and have a gap with students’ daily lives. 

In fact, CT can be seen in many different situations in life. For example, students decompose an exercise requirement to easier understand it. Students optimize their route (moving algorithm) from their houses to school. Thus, there is a question for curriculum writers to continue developing daily connection aspects into the current CS curriculum to help students expand their CT skills outside CS classes. 

In summary, after reviewing two curricula that use Scratch to teach CT, decomposition, algorithm, iteration, and generalization are CT elements, these two curricula create a lot of chances for students to develop through the coding classes. However, abstraction, data, and CT application in daily life are the challenging aspects that need more investigation to integrate into the CS class. Curriculum writers can also think about including these elements in other disciplines since these elements are hard to incorporate into the CS curriculum.

Assessment of Computational Thinking

Several assessment approaches have been created and evaluated during the previous two decades. However, there is a gap in our ability to effectively assess CT skills (Shute et al., 2017). There are two reasons for this lack of consensus. To begin with, there is no agreement on what CT is. This review of research provides a CT definition and six main elements. As a result, the curriculum should encourage the development of these characteristics in order to help pupils acquire CT. Similarly, to evaluate students’ competence in each component, the assessment should be based on these factors. Furthermore, the evaluation must be able to measure each component’s learning progression. “Computational thinking is not a binary state of there or not there at a single point in time, and any approach to assessment should strive to describe where a learner has been, is currently, and might go” (Brennan & Resnick, 2012, p. 23). Indeed, it’s not a question of whether or whether kids have that skill. Assessment should also go deeper enough to evaluate where students are in that learning progression. And it could even be better if it creates opportunities for students to learn and know what they should do next to improve their skills.

The second reason for the lack of widely accepted assessment is there is no single assessment method that can become a complete set of assessments. Each assessment method has its own pros and cons. For example, using automation tools to evaluate students’ code could access students’ understanding of CS concepts (Brennan & Resnick, 2012), but it could not assess the CT skills that students use to create those codes. Think-aloud interviews and debug tasks combined with observation are possible for assessing some aspects of CT (Brennan & Resnick, 2012; Fronza et al., 2017; Werner et al., 2012). It is important to note that this form of assessment is more time consuming, so it will be challenging to use these kinds of assessments widely. Multiple-choice assessment is also considered because of its convenience and scalability (Grover et al., 2014, 2015). Nevertheless, it is difficult to cover all aspects of CT with only multiple-choice questions as well as access the skillfulness level of each CT’s element. As a result, if teachers and curriculum writers want to have a comprehensive evaluation, they should combine different approaches based on their possibilities in doing those assessments. However, the combination of assessments needs to cover six elements of CT, which include decomposition, abstraction, data, algorithm, iteration, and generalization so that it can fully evaluate students’ CT skills.

Finally, there is a need for adding daily-life connection aspects to existing assessments. Learning to program has largely focused on coding skills, which has been criticized for lacking demonstration of how that knowledge can be applied to other subjects (Kafai & Burke, 2013). Since CS is only a tool to teach CT, CT assessment is not limited to the CS context. As a result, in addition to evaluating how students use CT in CS, it is also essential to evaluate their capacity to apply these abilities in other areas of life. The daily-life connection aspect must be included in order to ensure that students have those abilities and are prepared for a variety of careers in the future.

Conclusion and Implications

There is no doubt about the importance of computational thinking (CT) education for students’ future development (Mohaghegh & McCauley, 2016; Wing, 2017; Zhang & Nouri, 2019). Research has found that programming is widely used to develop CT skills (Shute et al., 2017). However, because there are still so many questions about what core characteristics of CT are, how they are related to computer science (CS), and how these skills are being taught and assessed in CS education (Nouri et al., 2020; Bocconi et al., 2022), there is a gap between CT and current CS education. Thus, this research aims to review current studies to find agreement on the definition of CT and its main elements. Next, two curricula, which thousands of students have used, are examined to see how their CT elements are related to CS education and what is still missing. Some limitations of current CT assessments and improvement suggestions are also included to help guide future work.

Wing (2017) has garnered the most agreement in defining CT. Computational thinking is “the thought processes involved in formulating a problem and expressing its solution(s) in such a way that a computer—human or machine—can effectively carry out” (Wing, 2017, p. 8). This definition sees CT as a thought process, which incorporates the initial ideas about CT as laid out by Papert in 1980. This definition also broadens the term CT outside the CS scope by defining solution implementation that can be done by humans or machines. Therefore, it maintains the essence of CT as a necessary skill for all professions. While opening the scope of the definition, it still shows that the root of CT is CS, which is the process of defining and solving problems. In conclusion, this definition includes all the ideas from the past to the present and from various perspectives. It should be widely used as a formal definition of CT on which all other research can be based, a solid foundation for the future development of CT education.

There are six main elements of CT: decomposition, abstraction, data, algorithm, iteration, and generalization. Computer scientists and education researchers mostly agree upon these elements because they cover all must-have skills used to form and solve problems effectively. As a result, CS curriculum designers should use these characteristics as a checklist when creating CS learning activities that assist students to grow all these skills and fully develop CT skills. 

The Harvard creative computing with Scratch and STEAM for Vietnam computational thinking and Scratch programming are two curricula that use Scratch programming language to support students learning about CT. Aligning these curricula with six elements of CT, highly supports decomposition, algorithm, iteration, and generalization. In detail, students learn about decomposition when analyzing big projects into characters and lists of actions. Students write algorithms to organize and optimize a character’s actions before coding. Students can try, test, and fix their programming, which allows them to practice iteration. And finally, when students re-use and re-mix the code, they use generalization skills to speed up their solution implementation. 

Despite these curricula’s advantages, abstraction and data are two elements that need more development because students have few chances to practice these skills. This might be the limitation of either the Scratch programming language or of the curriculum itself. Therefore, this problem needs more investigation to support students’ abilities in further developing their abstraction and data skills. In my opinion, the listed CT elements require advanced programming skills that are tough to demonstrate in a drag-and-drop programming language, like Scratch. Researchers also can think of moving these skills to a higher level of education and introducing them with text-based programming languages, such as Python and Java. However, text-based programming languages have a disadvantage: they are not as easy and fun as Scratch. This disadvantage might be a barrier for the majority of students to access CT. As a result, my recommendation is to integrate other subjects to support developing abstraction and data skills for K-9 students. The integration also supports students seeing and applying CT in all other aspects outside CS.

Since many researchers have agreed with CT’s elements mentioned above, they are guidelines for CT assessment. The researchers also explained that no single assessment method could comprehensively evaluate students. Thus, curriculum developers and teachers should use more than one assessment suitable for their teaching scope. For example, interviews combined with observation will be the best evaluation, but it takes time and requires a lot of skill from teachers. Whereas, multiple choice questions are not as efficient as interview and observation, but they are quick and easy to scale. Another factor to consider when creating future CT assessments is each CT element should have its learning progression; it is not as simple as whether or not kids have that skill. Assessments should show where students are on that progression and what is the direction for students to achieve a higher level in that skill.

Finally, the most significant mission for researchers in the future is to help students to use CT skills widely in all aspects of life. CT skills are proven as important as reading, writing, and arithmetic (Wing, 2006), but the question of why it is important to learn from students’ perspective persists. If students can not apply CT skills to their daily lives, CT education is meaningless. As a result, curriculum and assessment need to create more chances for students to connect and broaden their CT practices in class with CT practices in other aspects. Researchers need to fill the gap between CT in CS and other aspects to develop an efficient CT education. It can create more connections between CS education and students’ daily life. It can incorporate CT in multiple disciplines. It is essential to help students have CT skills, a skill set of the 21st century, to prepare for a successful future.

There are some limitations in this review of research. Only 25 studies and two curricula are carefully reviewed. The literature review is only focused on the CS aspect, especially programming, while CT is included in many disciplines. Thus, in future work, I intend to expand my work by evaluating the six core elements focused on in this paper in other aspects. In addition, I will continue to develop the STEAM for Vietnam curriculum and assessments to cover all core CT’s elements and help students connect CT skills that they have learned in CS class to their daily lives. 

References

Alsop, T. (2021, May 27). U.S. Tech employment by Occupation 2020-2030. Statista. Retrieved February 2, 2022, from https://www.statista.com/statistics/674463/united-states-electronic-components-manufacturing/#:~:text=In%202020%2C%20the%20United%20States,to%20reach%20almost%206%20million. 

Anderson, N. D. (2016). A call for computational thinking in undergraduate psychology. Psychology Learning & Teaching, 15, 226e234. http://dx.doi.org/10. 1177/1475725716659252.

Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48- 54.

Balch, C., Chung, M., & Brennan, K. (2014). Creative Computing Curriculum. Retrieved April 2, 2022, from https://creativecomputing.gse.harvard.edu/guide/curriculum.html 

Basawapatna, A. R., Koh, K. H., & Repenning, A. (2010). Using scalable game design to teach computer science from middle school to graduate school. In Proceedings of the fifteenth annual conference on Innovation and technology in computer science education (pp. 224-228).

Bell, T., Alexander, J., Freeman, I., & Grimley, M. (2009). Computer science unplugged: School students doing real computing without computers. New Zealand Journal of Applied Computing and Information Technology, 13(1), 20–29.

Bers, M., Flannery, L., Kazakoff, E., & Sullivan, A. (2014). Computational thinking and tinkering: Exploration of an early childhood robotics curriculum. Computers & Education, 72, 145e157. http://dx.doi.org/10.1016/j.compedu.2013.10.02.

Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York, Toronto: Longmans, Green.

Bocconi, S., Chioccariello, A., Kampylis, P., Dagienė, V., Wastiau, P., Engelhardt, K., Earp, J., Horvath, M.A., Jasutė, E., Malagoli, C., Masiulionytė-Dagienė, V. and Stupurienė, G. (2022). Reviewing Computational Thinking in Compulsory Education. Publications Office of the European Union, Luxembourg. ISBN 978-92-76-47208-7, DOI:10.2760/126955, JRC128347.

Brennan, K. (2022). Karen Brennan – scholar.harvard.edu. Retrieved April 2, 2022, from https://scholar.harvard.edu/files/kbrennan/files/brennan_cv.pdf 

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American educational research association, Vancouver, Canada (Vol. 1, p. 25).

Broughel, J. & Thierer, A. (2019). Technological Innovation and Economic Growth: A Brief Report on the Evidence. Mercatus Research, Mercatus Center at George Mason University, Arlington, VA.

Cansu, S. K., & Cansu, F. K. (2019). An Overview of Computational Thinking. International Journal of Computer Science Education in Schools, 3(1), n1.

Code.org. Glossary. (n.d.). Retrieved March 24, 2022, from https://code.org/curriculum/docs/k-5/glossary#:~:text=algorithm,steps%20to%20finish%20a%20task.

Code.org. (2022). Code.org Statistics. Code.org. (n.d.). Retrieved February 4, 2022, from https://code.org/statistics 

Code.org. (2013). President Obama asks America to learn computer science. https://www.youtube.com/watch?v=6XvmhE1J9PY&t=72s

Code.org. (2022). Quotes on computer science – what leaders and trendsetters have to say. Code.org. (n.d.). Retrieved April 1, 2022, from https://code.org/quotes 

Code.org. (2021). State of CS Report: Accelerating Action Through Advocacy. Code.org. (n.d.). Retrieved February 20, 2022, from https://advocacy.code.org/stateofcs 

Dewey, J. (1923). Democracy and education: An introduction to the philosophy of education. Macmillan.

ISTE (The International Society for Technology in Education) & CSTA (Computer Science Teachers Association). (2011). Operational Definition of Computational Thinking for K-12 Education. http://www.iste.org/docs/pdfs/Operational-Definition-of-Computational-Thinking.pdf

Foshay, A. W. (1991). The Curriculum Matrix: Transcendence and Mathematics. Journal of Curriculum and Supervision, 6(4), 277-93.

Fronza, I., Ioini, N. E., & Corral, L. (2017). Teaching computational thinking using agile software engineering methods: A framework for middle schools. ACM Transactions on Computing Education (TOCE), 17(4), 1-28.

Grover, S., Cooper, S., & Pea, R. (2014). Assessing computational learning in K-12. In Proceedings of the 2014 conference on Innovation & technology in computer science education (pp. 57-62).

Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer science course for middle school students. Computer Science Education, 25, 199e237. http://dx.doi.org/10.1080/08993408.2015.1033142.

Hemmendinger, D. (2010). A plea for modesty. ACM Inroads, 1(2), 4-7.

Kafai, Y. B., & Burke, Q. (2013). The social turn in K-12 programming: Moving from computational thinking to computational participation. Paper presented at the Proceedings of the 44th ACM Technical Symposium on Computer Science Education (pp. 603–608), Denver, CO, USA.

K-12 Computer Science Framework Steering Committee. K–12 Computer Science Framework. (2016). Retrieved from http://www.k12cs.org.

MIT Lab. (2022). Scratch Statistics. Scratch. Retrieved February 20, 2022, from https://scratch.mit.edu/statistics/ 

Mohaghegh, M., & McCauley, M. (2016). Computational Thinking: The Skill Set of the 21st Century. International Journal of Computer Science and Information Technologies (IJCSIT), 7(3) ISSN: 0975-9646, pp. 1524-1530.

National Research Council. (2010). Committee for the Workshops on Computational Thinking: Report of a workshop on the scope and nature of computational thinking. Washington, D.C: National Academies Press.

Nouri, J., Zhang, L., Mannila, L., & Norén, E. (2020). Development of Computational Thinking, Digital Competence and 21st Century Skills When Learning Programming in K-9. Education Inquiry, 11(1), 1-17. http://dx.doi.org/10.1080/20004508.2019.1627844

Papert, S. A. (1980). Mindstorms: Children, computers, and powerful ideas. Basic books.

Rahman, M. (2019). 21st-century skill ‘problem solving’: Defining the concept. Asian Journal of Interdisciplinary Research, 2(1), 64-74.

Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142-158.

Sneider, C., Stephenson, C., Schafer, B., & Flick, L. (2014). Exploring the science framework and NGSS: Computational thinking in the science classroom. Science Scope, 38(3), 10.

STEAM for Vietnam (n.d.). CS 001 – Computational Thinking and Programming with Scratch. STEAM for Vietnam. Retrieved April 2, 2022, from https://steamforvietnam.org/courses/advanced-scratch 

U.S. Bureau of Labor Statistics. (2021). Computer and Information Technology Occupations: Occupational Outlook Handbook. U.S. Bureau of Labor Statistics. Retrieved February 2, 2022, from https://www.bls.gov/ooh/computer-and-information-technology/home.htm

U.S. College Board. (2022). AP computer science principle. Retrieved March 6, 2022, from https://apcentral.collegeboard.org/courses/ap-computer-science-principles/course

Werner, L., Denner, J., Campe, S., & Kawamoto, D. C. (2012). The fairy performance assessment: Measuring computational thinking in middle school. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (pp. 215-220).

Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33-35.

Wing, J.M. (2017). Computational thinking’s influence on research and education for all. Italian Journal of Educational Technology, 25(2), 7-14. DOI: 10.17471/2499-4324/922

Yadav, A., Hong, H. & Stephenson, C. (2016). Computational Thinking for All: Pedagogical Approaches to Embedding 21st Century Problem Solving in K-12 Classrooms. TechTrends 60, 565–568. https://doi.org/10.1007/s11528-016-0087-7

Zhang, L., & Nouri, J. (2019). A systematic review of learning computational thinking through Scratch in K-9. Computers & Education, 141, 103607.

Appendix A

Rubric for Assessing Development of Computational Practices in the Harvard Creative Computing Curriculum

EXPERIMENTING AND ITERATING	LOW	MEDIUM	HIGH
Describe how you built your project step by step.	Student provides a basic description of building a project, but no details about a specific project.	Student gives a general example of building a specific project in a certain order.	Student provides details about the different components of a specific project and how they were developed in a certain order.
What different things did you try out as you went along with your project?	Student does not provide specific examples of what s/he tried.	Student gives a general example of trying something in the project.	Student provides specific examples of different things s/he tries in a project.
What revisions did you make and why did you make them?	Student says s/he made no revisions, or only states s/he made revisions but gives no examples.	Student describes one specific revision s/he made to the project.	Student describes the specific things s/he added to the project and why.
Describe different ways you tried to do things in your project, or when you tried to do something new.	Student provides no examples of trying something new.	Student provides an example of trying something new in the project.	Student describes specific new things s/he tried in a project.
TESTING AND DEBUGGING	LOW	MEDIUM	HIGH
Describe what happened when you ran your project that was different from what you wanted.	Student does not describe what was different when s/he ran the project from what s/he wanted.	Student describes what went wrong in the project, but not what s/he wanted it to do.	Student gives a specific example of what happened and what s/he wanted to have happen when s/he ran the project.
Describe how you read through the scripts to investigate the cause of the problem.	Student does not describe a problem.	Student describes reading through the scripts but does not provide a specific example of finding a problem in the code.	Student describes reading through the scripts and provides a specific example of finding a problem in the code.
Describe how you made changes and tested to see what happened.	Student does not describe what problems s/he had or the solution.	Student provides a general example of making a change and testing it out to see if it worked.	This student provides a specific example of making a change and testing it out to see if it worked.
Describe how you considered other ways to solve a problem.	Student does not provide an example of a solution to a problem.	Student provides a general example of a solution to the problem.	This student provides a specific example of a solution to the problem.
REUSING AND REMIXING	LOW	MEDIUM	HIGH
Describe if/how you found inspiration by trying other projects and reading their scripts.	Student does not describe how s/he found ideas or inspiration from other projects.	Student provides a general description of a project that inspired him/her.	Student provides a specific example of project that inspired him/her and how.
How did you select a piece of another project, and adapt it for your project?	Student does not describe how s/he adapted scripts, ideas or resources from other projects.	Student identifies scripts, ideas or resources s/he adapted from other projects.	Student provides specific examples of scripts, ideas or resources s/he adapted from other projects and how.
How did you modify an existing project to improve it, or enhance it?	Student does not describe modifying another project.	Student provides a general description of modifications s/he made to another project.	Student provides specific examples of modifications s/he made to other projects and why.
How did you give credit to people whose work you built on or are inspired by?	Student does not give credit to others.	Student names people whose work inspired him/her.	Student documents in project and/or on the Scratch website the people whose work inspired him/her.
ABSTRACTING AND MODULARIZING	LOW	MEDIUM	HIGH
How did you decide what sprites are needed for your project, and where they should go?	Student provides no description of how s/he selected sprites.	Student provides a general description of deciding to choose certain sprites.	Student provides a specific description of how s/he made decisions about sprites based on goals for the project.
How did you decide what scripts are needed for your project, and what they should do?	Student provides no description of how s/he created scripts.	Student provides a general description of deciding to create certain scripts.	Student provides a specific description of how s/he made decisions about scripts based on goals for the project.
How did you organize the scripts in ways that make sense to you and others?	Student does not describe how s/he organized scripts.	Student provides a general description of how s/he organized the script.	Student provides specific examples of how s/he organized the script and why.

Note: Adapted from Harvard Creative Computing Curriculum. Balch, C., Chung, M., & Brennan, K. (2014). Retrieved April 2, 2022, from https://creativecomputing.gse.harvard.edu/guide/curriculum.html 

Appendix B

Example multiple-choice questions in STEAM for Vietnam Computational Thinking and Scratch Programming curriculum

Question 1: An algorithm can simply be understood as the list of steps in order to do a task. Please arrange the following steps to form an algorithm for cooking rice.

1. Press the start button on the rice cooker
2. Put water to cover all rice
3. Wash the rice
4. Put rice in a rice cooker

1. 1 🡪 2 🡪 3 🡪 4
2. 3 🡪 4 🡪 2 🡪 1
3. 3 🡪 2 🡪 4 🡪 1
4. 1 🡪 3 🡪 4 🡪 2

Question 2: How many sprites are on this screen?

1. One sprite: Cat
2. Two sprites: Cat & Grill Fist
3. Two sprites: Cat & Maze
4. Three sprites: Cat, Grill Fist, and Maze

Question 3: When running the following program, what will the Cat say?

1. I love math!
2. Hmm…
3. I love math! Then Hmm…
4. Nothing

Appendix C

Example Rubrics to support Students’ Peer Assessment in STEAM for Vietnam Computational Thinking and Scratch Programming curriculum and Scratch Programming curriculum

Project: Hurry Cat

Project requirements:

1. All sprites are displayed in the correct positions
2. The cat can move around when the player presses the arrow buttons
3. The can can’t move through the wall of the maze
4. The cat cheers when touching the grilled fish

Grading Levels:

• Not pass: Do not meet all the above requirements or not being playable
• Pass: Meet all the above requirements 
• Excellent: Meet all the above requirements as well as add upgrade functions