Helping you make games for crowdsourcing & citizen science
Helping you make games for crowdsourcing & citizen science
last updated October, 2018
Games for science and crowdsourcing include any game or videogame used to help produce science, or in the case of crowdsourcing, any productive output involving more than one person. Such games are seeking to produce, annotate, and/or analyze data as a result of play. Often such games are used in science, especially citizen science, that involves lots of users collecting data, but crowdsourcing games can be used in many productive situations that don't always deal with science.
No, crowdsourcing games could be used to produce ideaa for a business or non-profit like World Without Oil did. Some games could be used to annotate collections of data used by libraries as Dartmouth's [XLAB] did for their project for the Institute for Libraries and Museum Science. Other games could be used to crowdsourcing AI for game companies.
Tell me more about games for science aside from citizen science games?
The games for crowdsourcing field is dominated by science games. However, in the large field of games for science, while a lot of focus is on games that help produce, annotate, and/or analyze scientific data, there are other game types, such as games for science education, promotion, recruitment, and more. One particularly interesting niche genre of games for science are games designed to put users into specific physical and/or emotional states (e.g. frustration) so subjects can be analyzed not on their gameplay but on changes in their physical and mental conditions as induced by the game.
Games can be used for many things, and in the field of science, not every game is citizen science or crowdsourcing oriented.
Citizen Science specifically means involving general members of the public in a collaborative effort to collect, annotate, and/or analyze scientific data. However, not all crowdsourcing games need to involve the public per se, some games could be aimed at very specific experts, or other non-public populations in efforts that are not necessarily as open to all. When games are targeted at a specific, specialized, player-base they may have many of the same mechanics, and systems as general citizen science games, but they are not as open as citizen science games are generally recognized to be.
Crowdsourcing games be they for citizen science, or otherwise, all have a basic structure. They tend to be client-server games where players access and play a game experience that records their gameplay in some form that results in new information. That information could be insights into the existing data exposed to the player (e.g analysis), additional data such as text descriptions of an image a user saw, or changed data such as corrections to inaccurately scanned text documents.
In all cases the goal is to present the player with gameplay that attracts them to a particular task, and incentivizes them to complete the work necessarily to capture the data desired. Ideally, such games would present gameplay that is fluid, productive, and augments the player in such way that the experience for the player is better than a non-game means to complete the same task. Another important best practice for such games is that they are able to educate the player and return to the player some value vs. just extracting value from the player community. For example a game about analyzing data concerning a historical event might also teach the user about that historical event and not just have them leave the game without knowing anything about the related work, project, or purpose.
Assuming a game can produce the data output required of its creators, that data is then used by the creator to accomplish the larger goal of the project, be it scientific discovery, organizational planning, public policy, marketing goals, etc. In a number of cases data from crowdsourcing games is used to inform machine learning models that when trained can improve their own ability to do the work instead of the original human players. Even then, humans, playing games, might be further engaged to help with edge cases machine learning models continue to fail at, and can even work in conjunction with machine learning systems to tackle problems together.
The field of crowdsourcing games has seen the use of a number of descriptive phrases including "Games with a Purpose" and "Human Computation Games" They are often the same thing, using games, to source human skill in the creation, annotation, and/or analyzation of data.
Gamification is a term of art and often marketing, that is used in inconsistent ways to describe a fusion of gameplay and task. It's important to be careful when using this term to begin with because it's often conflating two very different manifestations of games used beyond entertainment.
Let's be super-clear about gamification as a term to start.
Gamification in its most specific sense refers to adding simple elements often associated with games (e.g. leaderboards) but also with basic industrial and consumer psychology to add additional points of engagement and incentive to otherwise rote tasks and behaviors. For example, by creating a points scheme for correctly identifying the dominant object in an image that matches the majority of previous respondents, you could create a "game" or contest. This system incentivizes users to compete at identifying objects in photographs such that they do it longer and analyze more photographs then if merely prompted for an answer with no points awarded or other elements of gameplay such as leaderboards.
A larger full GAME effort might be much more involved than its more app-like "gamification" solution. It could involve deeper levels of puzzle and strategy, and often, many more trappings that create a larger videogame experience.
To people who work in these fields, gamification and games are not exactly the same thing.
Rather than get completely caught up in these sorts of design wars, consider the idea of a gamut that runs as follows:
app/task gamification full game
When something is plainly an app or task, it's an assignment with few trappings designed to foster gameplay amongst the users. It's simply a prompt, and perhaps some means to help accomplish the requested task. "Please tell me the name of the object seen here." Where as a gamification approach might be a simple app experience but each right answer gets you 1pt and 5-in-a-row gets you an added bonus of 10pts and the players with the best scores compete for leaderboard fame, or even real-world incentives. All of which is designed to add a bit more incentive and productivity to the underlying task. Thus the user experience is moving a bit more away from app/task and bit more toward full game. Finally, imaging that many other elements are added, and players are asked to do other elements such as sort things into categories, or play a matching game that looks and feels much more like a complex card game with multiple rules-of-play than just answering a question of what do you see? Now we're much further across to the right into something that looks, feels, and plays like the games we try to play everyday. There is perhaps more strategy at work, additional systems to understand and master, all while accomplishing the underlying data goals of the create of the game.
Having explained the differences between games and gamification the answer to this question is that not all crowdsourcing games are "gamification" instances. Yes, many do fall closer to the left side of the described gamut than the far right side, but that doesn't mean your crowdsourcing game has to. It all depends on the goals you have for your project, and how easy, and beneficial it is to build deeper forms of gameplay experiences to achieve the goals you have.
In the games crowdsourcing and citizen science field there are many terms that get thrown around. It's important to know what some of these are, and the most specific definition ascribed to them because such terms often get misused, and as such leads to confusion and conflation of what a project's approaches and goals might be.
Collective Intelligence : This refers to users who are working together to solve a particular challenge and as such can rely on the collective effort and cognition of all who are participating.
Wisdom of Crowds : This refers to the idea that by have the ability to observe independent behavior of a group of users who aren't influencing each other. By observing the choices they make, activity, and answers in a game environment, a system could potentially help surface new information and analysis using game-based crowdsourcing.
What's important to understand is that these terms are NOT the same thing, and often are used interchangeably to mean the same thing (often the collective intelligence definition) and thus it can be confusing. Henry Jenkins of USC has a great discussion of these terms, especially in videogame contexts, on his blog.
Yes. As with many new fields, and fields that are at the intersection of multiple disciplines this is not uncommon. It's not the end of the world either. However, as the field begins to rally around specific terms, and their most specific definitions, that clarity will help the field grow.
There are many reasons games are a critical go-to element and solution for many crowdsourcing and citizen science projects. While it's often cited that games are fun, and thus attractive and motivating to users, these are not the only reasons games are used in crowdsourcing and citizen science.
In one sense, games are essentially problem solving environments, where often, the player is given the means to solve a problem (e.g. in Super Mario jumping is the means) but isn't always given the answer to a particular challenge. This means game designers are uniquely good at crafting experiences where a solution isn't a given, but the process to solve it is. By optimizing this experience games offer players not just motivation and engagement, but also information architecture, and user experience, that aids them in thinking about and accomplishing the solution to a problem. This can include uniquely designed visualizations of data, interesting rules that channel player behavior, and experiences designed to efficiently tutor a player to expert level.
While it's easy to state games can motivate people, do not overlook the many deeper things games do to aid a player in solving a problem without doing it for them. Great games that do this often are games where the true efficiency to solving a problem are unknown or so vast that a single algorithm isn't going to do the job as well as expert players and player communities. In such cases, game-based crowdsourcing, should lead to a form of comparative advantage toward solving a problem.
An underlying element to why games are useful for crowdsourcing and citizen science is that games often create environments and processes of play that make complex tasks, or tasks that require learning some expert skill, possible to people not specifically, or originally trained to do a task. Few if any of the players of SimCity had ever run an actual city before and thought about optimal designs for public transportation. Few people playing Kerbal Space program ever have launched a real spacecraft into orbit, yet both games have seen interesting and previously unknown expertise formed through playing these games. By applying that same idea to crowdsourcing and citizen science games, game designers, can create systems that can help ordinary users, gain and improve their expertise in an underlying subject area or task specialty such that they find, and/or generate a population of players that exceeds that of previously recognized experts. In fact, this is what's happened with games like EyeWire and Fold.it
There are many ways games can benefit crowdsourcing and citizen science efforts, so long as you think beyond the most common ideas, you have the chance to harness deeper capacities of games and game design to aid your specific project.
There are several ways to think about what the core uses of crowdsourcing games are. First, AI Collection Puzzle/Challenge Work (wisdom of crowds) Puzzle/Challenge Work (collective intelligence) Ideation and data collection
You can also think about what users are doing Creating data : Transforming data : Augmenting data :
Processing Human-in-Loop Human, Computer Participatory for Engagement
In human-in-loop processing, the human is necessary to computation for specialized capabilities, for machine donated resources crowdsourcing is used to gain access to CPUs, power, and storage necessary to crunch the large amount of data.
Participatory for engagement means that humans are helping process data but not because the computer isn’t capable but because there is a need to engage people in the process for alternative outcomes.
Observable Gameplay
Semantics & Natural Language Processing, Social Graph
Observable crowdsourcing means that the players actions are observed and sourced as data toward a higher-end outcome (e.g. AI opponents, language parsers). Gameplay is useful especially for AI, Semantics and Natural Language Processing gains from human interactions (e.g. see Restaurant Game). Finally observing social graphs can help gain additional crowdsourced data.
Physical
Environmental/Geolocation, Data Capture
"Capture Physiological Data"
Capture Transactional Data Physical crowd models distinguish themselves by capturing data that requires humans to produce or capture. Environmental and geolocation data can involve photos, flora/fauna samples, 1st person observations, periodic mobile sensor readings, etc. Physiological data is self-report, or sensor captured biometrics and emotional health reports. Transactional data while possibly captured through computer networks still requires a real-world human decision to initiate the underlying transaction.
No. While everyone wants more players, and in many cases more players can help achieve higher rates of productivity, what has been found by many projects is a dedicated band of even a few hundred or thousands of players can be more-than-enough to achieve meaningful results.
The dedicated part is important though, experienced players are critical for many game communities, not just crowdsourcing games. It is these players which often lead the community, teach new players, and offer advice to them as they gain in experience.
Good players are essentially experienced players, who not only may be cognitively great at the puzzles, tasks, and challenges your project presents to people, but their ability to dedicate large hours, and return consistently over time makes them super important. It's that combination that's critical.
Fostering a game community takes work, and within crowdsourcing it's doubly important because often the gameplay while interesting to the most dedicated players, is not eventually the main reason they keep coming back. Building systems and activity that bonds players to your project, to each other, that provides recognition for contributions, and gives back to players through education, recognition, new challenges, community outreach and more, are important elements of successful games for crowdsourcing and citizen science.
The ideal games offer good gameplay, that respects the player, and has a fluid, fun feeling process of play. Good gameplay is a relative term, as many crowdsourcing and citizen science games may be rooted around tasks and challenges that are not necessarily intrinsically fun. The goal is to maximize what you can.
The ideal games give the player the ability to use and explore strategies that are compelling decisions for them to execute. Games which go beyond rewarding points for performance, and which offer the chance to explore systems, and different approaches toward a larger set of goals and opportunities are going to provide better game experiences than those which simply prompt the user for a performance and then quickly provide a points-and-achievement evaluation to their work.
A fun game alone is not the only ingredient for a good crowdsourcing and citizen science game. As mentioned earlier, creating community, and leaders, is also critical, as is onboarding new players, and getting them to become great contributors to the game. A final element is that good crowdsourcing and citizen science games GIVE BACK to their players. Beyond providing community, they ideally educate their users, and help them to learn more about the projects and related elements that they are contributing to. Often it's easy to see players as the end node of a large process, but they should really be part of an active loop of producing and receiving.
Hopefully you will! In most health game project cases to date a clear sponsor funds the creation of a game, be it by grant, or as part of their cost of doing business. A large swath of the serious games space overall (including work on health games) is a work-for-hire services-based development industry. Clients produce problems and capital, developers build games they hope offer a solution. Yes, in a number of rising cases, developers are raising funds and/or building health games first, and attempting to commercialize them but aside from some key markets (e.g. exercise) these efforts are not broadly applied.
You may have heard that many health games are funded through grants. This is true to an extent, many health games are part of research or public interest funding provided by governments and philanthropic foundations. Many others are funded by enterprises for their internal user (e.g. training) or external use (marketing and customer support) and only a handful so far are funded as commercial products for sale to target users.
The bottom line is you will need to secure the funding for your game. Most developers are fee-for-service companies and are not looking to build games, however interesting they may be, on spec. If you haven't already secured funding you will want to look into various forms of financing for games be they grants or other means of raising funds.
All games need to promote themselves, thankfully for crowdsourcing, and especially citizen science games there is a growing community of sites that promote these opportunities and help you attract players. Players might also come from using systems like Amazon's Mechanical Turk service, and recruitment partnerships with underlying communities that are interested in or affect by the topic or challenge the game is seeking to address. For example, a game that seeks to help with a particular disease space might partner with the various non-profits that advocate and assist patients with the disease or condition.
Good PR will also involve connecting to communities of game players via the gaming press, and letting them know that a new novel challenge awaits them. Additional mainstream press will be interested in the novelty of the project.
Despite there being a growing number of such projects the appetite for covering games that have larger callings and effects is still strong.
In some cases yes. However, there are many reasons to build crowdsourcing and citizen science games that make them useful even if AI can reasonably address a particular task.
For example:
* Not all AI algorithms are actually great at solving some of the harder tasks humans are good at. Edge cases can abound. Even when an AI system is good, humans may still be better, or best at finding alternative solutions. This is why some human computation games still keep running. Sometimes they work in conjunction with a growing intelligent AI.
* The computing power required for some AI systems may be expensive and not as available. One of the useful elements of crowdsourcing and citizen science games is that it also spreads out the cost for the computer time and electricity resources to the edges of the network on each individual player's machine.
* Humans are capable of easy and significant mobility. If your game requires activities that require mobility there is almost no other way to do it.
* Crowdsourcing games could be collecting opinions, and ideation, not the sorts of things AI will help with.
* By involving people in citizen science and crowdsourcing projects, even if there are alternative ways that might be more efficient, often fail to help improve public understanding and appreciation for the issues. Not every game need to be about optimally solving the problem. There are additional benefits to participatory exercises.
These are just a few examples of why building crowdsourcing and citizen science games goes beyond the notion of an optimized problem solving machine. The more you think holistically about all the outputs and benefits of a game, the better.
Looking at crowdsourcing and citizen science games from this angle, there are three types of problems worth thinking about.
First, is whether the problem is best solved using a game. This especially includes if you can design a good enough game around the tasks and challenges the player is being asked to address. This certainly includes an assessment of if humans can better and more optimally solve a particular problem than a computer. However, it doesn't always have to be about humans being entirely better than a computer. As previously discussed, there are additional benefits that can be captured even when an AI-based problem solving approach might work.
If a game isn't well suited to the problem, or the benefits of human computation and performance aren't as strong as you'd like, the problem is that the experience might be sub-optimal enough as to fail.
Thus one problem is to ask, are games are good matches? Given the interest in crowdsourcing and citizen science games one problem that exists is that it's now a popular idea to use games in these problem spaces -- without giving much though to the larger issues and consequences.
The second problem arises from games that do little to appreciate the players and their communities. As discussed throughout this FAQ, the best crowdsourcing games are two-way streets where there is some level of value flowing and supported in both directions between the game and the game-based labor supporting it.
It is a problem, when any crowdsourcing or citizen science project takes their player-based for granted. So don't do it.
There is much to know to learn how to make any game let alone one a health related game. Health games lay at the intersection of many domains of knowledge including game design, software development, psychology, cognitive science, behavior change, biology, and much more. Health game projects have lots of unique features that make their development processes, while similar to entertainment games, different. Depending on your role in the process of creating a health game it is helpful to understand the basics of making games, providing input into production, and managing creative, iterative productions.
Since so many health games deal with issues around behavior change and persuasive health technologies it is also a good idea to study up on these areas of activity. Whether it's to understand ideas like social learning theory, message framing, behavioral economics, or the theory of planned behaviors, the last few decades have produced a plethora of evidence-based theories about learning, and behavior change that is critical to the creation of successful health games. This knowledge is not always held by developers, or it's understood, but not in a formalized manner even though many game design philosophies do dovetail with successful learning and behavior change practices.
Beyond the technical knowledge, and the domain knowledge relevant to what your game is about, the most important thing to know to make health games is working with teams of creatives, and how to capture constructive feedback that you can feed back into the game development process.
All-in-all what you need to know to make a good health game is how to identify, and eventually supplement across the team of professionals building your game the various knowledge bases that you bring together to make a great health or healthcare game. This can be a little bit of everything, but often includes the core domain knowledge of the game's subject, and the evidence-based theories that drive outcomes for your problem area.
Games research is quite a robust and ongoing area of activity. Hundreds of university based projects exist, many robust university programs focus exclusively on looking at the impact and capacities of games across a range of sectors.
There is a growing body of work around the use of games for purposes beyond entertainment especially in education, and health, to cite and build upon. Some of it is contained in journals that focus on games, game technologies, and serious games, however, it's important to know that a lot of work on the subject is also published in fields specific to the topic of the game and/or the problem it's addressing. This is especially true for health games where there are many health journals that are the preferred choice to publish in. This is why when working to find prior work, and evidence you will want to look not only for generalized games research, but also specific research within the field/topic your game is targeting.
To pinpoint several papers or research projects that may be useful can be difficult but the resources on GamesandCrowds.Tips and SeriousGames.tips can help you look for research, games, journals, and conferences that may be useful for your efforts.
On GamesandCrowds.Tips are many links to useful resources to follow up with via the Web, social media, books conferences, and video. Additional links and resources can also be found on our sister-site SeriousGames.Tips.