2017 Conference Sessions

Check out the various panel, paper, and roundtable sessions that will be held at this year's conference.

Thursday, July 20, 2017

Opening Plenary | 8:30am - 10:00am

Keynote Speaker: Marika Shioiri-Clark, SOSHL Studio

Trained as an architect, Marika is a leading ambassador and practitioner of human-centered design, and, in 2011, was a global fellow at design nonprofit IDEO.org, working on projects in the education, health, and sanitation sectors in the United States and Ghana. Currently, Marika is working with the International Rescue Committee to design and pilot an early childhood development program for refugees in the Syrian response region. Her current practice relates to human-focused, research-driven design that considers not only the specific needs and goals of people who will use spaces, but also the role spaces and message construction can play in issues related to power, authority, and access, as well as how community-centered approaches can effect positive social outcomes.

Concurrent Sessions | 10:15am - 11:30am

Exploring and Examining Art Museum Multi-visit Program Evaluations
Elizabeth Bolander, Michelle Grohe

This panel presentation will provide an in-depth look at two art museum case studies of multi-visit programs representing different ends of the Pre-K to high school spectrum and how project-specific research can spark broader conversations about evaluation institution-wide. An interactive discussion about multi-visit and longitudinal evaluations will follow.

Beyond Excellent: The Overall Experience Rating
Nick Visscher, Andrew Pekarik, Kerry DiGiacomo, Hannah Ridenour

Unsatisfied with common measures taken from the business world? This session is for you! The Overall Experience Rating can be used to evaluate and compare across museum experiences. Hear from a panel of OER evangelists about how it's been adopted in a variety of sites from art museums to zoos.

Multi-Site Evaluations: Evaluation Capacity Building and Mutually Beneficial Partnerships
Gretchen Haupt, Sarah Cohn, Jen Gathings, Ryan Auster

Multi-Site evaluations are typically designed to best serve the needs of the individual or organization leading the evaluation; this session will offer examples of mutually beneficial evaluations that focus on building evaluation capacity at participating sites by training local staff and volunteers to assist with data collection.

Emerging Methodologies: Organizational Autoethnography and Duoethnography in Museum Evaluation
Ann Rowson Love, John Boda, Victoria Eudy

Exploring new pathways includes applying new methods for understanding museum practices including training emerging professionals. This session introduces and makes a case for using organizational autoethnography and duoethnography as evaluation methods for analyzing education and curation practices. Presenters unpack the methods and engage participants in sample data collection activities.

Applying Learning from Multi-year Studies to Improve Zoo Visitor Experience
Shasta Bray, Dolly Hayde, Manda Smith, Nadya Bennett

Speakers will share how multi-year studies conducted at three Midwestern zoos informed improvements in visitor engagement through the practical application of findings to the design of exhibits, programs and general visitor experience considerations. Large group discussion on conducting and building buy-in and administrative support for multi-year studies will follow.

Concurrent Sessions | 1:15pm - 2:30pm

PAPERS: Methods and Methodologies

What to Do After Piloting Has Yielded a Null Result
Gloria Segovia, Aaron Price

The Power of Mixed Methods: An Assessment of Visitor Emotion
Jana Greenslit

Shaping Visitor Experience Using Confessional-Booth-Style Video at Expo Events
Kelley Staab

Beyond Our Walls: Museum Evaluation Providing Voice for Civic Change
Shana Hawrylchak, Betsy Loring, Emmy Hahn, Robert Ryan

Presenters will discuss two evaluations of the EcoTarium's City Science exhibit that provided opportunity for broader civic collaborations. The first, an embedded evaluation tool, led to collaboration with urban planners at UMass Amherst. The second, a more traditional front-end evaluation, provided implications towards use by state and city planning agencies.

Found in Translation: Building Capacity for Multilingual Data Collection & Analysis
Ivel Gontan, Cecilia Garibay, Verónica Betancourt

Are you interested in learning more about data collection in multiple languages? This session will focus on processes and challenges for collecting and analyzing data in multiple languages. Session will build capacity around: cultural competency, research planning, and staying true to the original meaning of the data.

Key Social Science Messages for Understanding Visitor Studies
Deborah Wasserman, Joe Heimlich, Dolly Hayde, Elaine Horr

What messages succinctly communicate the science behind visitors studies and the nature of our work? Together, visitor studies veterans and newcomers will build a list from 10 key messages produced by an IMLS-funded project "Social Science Under the Microscope: messages and prototypes for communicating social science as a science.

Children's Museum Research Network: A Case Study in Collaborative Research
Sarah May, Nicole Rivera, Kari Nelson, Susan Foutz

Four members of the Children's Museum Research Network (CMRN) will describe how they conduct collaborative research - from shared responsibility in data collection to group data analysis. Panelists will engage audience members in collaborative meaning-making practices and analysis that will simultaneously reflect on the challenges and benefits of collaborative research.

Roundtables and Fireside Chats | 3:00pm - 3:45pm

Fireside Chat with Kirsten Ellenbogen

Kirsten Ellenbogen worked as a researcher and evaluator for many years but now serves as the President and CEO of Great Lakes Science Center in Cleveland, OH. Chat with Kirsten about her transition from museum researcher to CEO and learn about her ideas for new pathways in visitor studies based on her experience and position now.

Nurturing School Partnerships through Exhibit Evaluation
Julia McAllister, Joseph Wisne, Courtney Heppner

Learn how to enhance your institution's school partnerships through exhibit evaluation. Presenters will use a case study as a reference and provide practical advice to participants. Panel participants include representatives from a Columbus area school district and Roto, a design and fabrication company.

Playing with History: Evaluating Historical Thinking in Young Children
Allison Cosbey

While some believe young children are not developmentally ready to have meaningful experiences with history, others argue that their experiences can lay the foundation for future history learning. Join us to discuss frameworks for assessing historical thinking and their strengths and weaknesses for describing young children's interactions with history.

Making Sense of Evaluation and Visitor Studies: An Emerging Conversation
Kathy Kiser, Erin Shoffstall, Manda Smith

Visitor studies, evaluation, learning research, audience research. What do these terms mean? How are they similar? How are they different? Why does it matter? The presenters work in informal learning organizations with multiple research and evaluation teams. Learn about their experience and join the discussion to further examine this topic.

Examining Cultural Assumptions: Implications for Equity in Museums
Lindsay Maldonado, Nicole Rivera, Ed.D.

This session will explore cultural assumptions inherent to museums. Literature will be used to present alternative perspectives and challenge our ideas about culture and learning. Discussion will address the role of research in identifying and addressing cultural assumptions in museums and how researchers can work to create equity.

Concurrent Sessions | 4:00pm - 5:15pm

PAPERS: Understanding and Measuring Our Impact
Stephen Ashton, Ph.D., Alan Brown, Kimberlin Sturgis, Ph.D., Lei Zhao

It has become increasingly important for informal learning institutions to demonstrate the impact they have within their communities. In this session, presenters from a variety of organizations will share how they measure the impact they have on their visitors and communities. Q&A and discussion about understanding and measuring impact will follow.

Absent Visitors: New Approaches to Researching and Implementing Social Inclusion
Naomi Haywood, Sarah Lee

There is growing awareness across visitor studies that museums need to broaden their visitor profile. This presentation attends to this need by discussing ethnographic approaches that capture socio-cultural resources of under-represented visitors. It critiques approaches that focus on 'deficits' of such visitors, and promotes new ways of exploring and implementing social inclusion.

The Path to Assessing Student Learning Requires Planning and Perseverance
Amanda Krantz, Emily Holtrop, Missy Higgins-Linder

Fieldtrip programs are the primary ways museums serve school students (and in great numbers!), but assessment of student outcomes is often cursory because students are difficult to access. In this session, we will share lessons learned from a national study of art museum fieldtrips.

Identity Matters: "Science Identity" in Exhibitions, Evaluation, and Visitor Experience
Erin Wilcox, Gemma Mangione, Erica Kelly

How do science identities—the socially structured meanings different people make from science experiences—impact museum visitors? More broadly: How can professionals use theories of inequality to diversify exhibition design, evaluation, and visitor experience? We'll explore these questions discussing a "citizen science" exhibition at the San Diego Natural History Museum.

Promoting Conservation Learning and Behaviour Change in Zoos and Aquariums
Roy Ballantyne, Jan Packer, Karen Hughes, Judy Mann

The session discusses an empirical model of environmental learning, techniques for measuring behavior change, and research findings regarding the personal and experiential factors that influence behavior change in zoos and aquariums. On-site and post-visit interventions that encourage visitors’ adoption of environmentally responsible behavior are presented and their effectiveness evaluated. The presentation will discuss research undertaken in Australia, USA, Canada and South Africa.

Friday, July 21, 2017

Plenary Speaker | 9:00am - 10:00am

Invited Panel: Staff from The PAST Foundation

The PAST Innovation Lab is an Education R&D facility based in Columbus, Ohio, and its major goal is improving education by promoting research-driven, transdisciplinary problem-based learning (TPBL). Rooted in the legacy of the larger PAST Foundation, the Innovation Lab brings anthropologists, other researchers, and educators together to support learning experiences at the intersection of formal learning and community. Meanwhile, the Knowledge Capture division of PAST provides external evaluation services to schools and districts throughout the country, with a culturally relevant mixed methods approach that reflects the Foundation's overarching strategy of combining Anthropology with STEM to support learners.

Concurrent Sessions | 10:15am - 11:30am

PAPERS: Institutional Change

Examining Program Evaluation Practice and Capacity in Museums: A New Conceptual Framework
Agnieszka Chalas

Visitor Motivation as a Roadmap to Creating and Marketing Experiences
Tiffany Leason, Mindy Cultra

Look at Art. Get Paid: Accessibility and Cultural Critique
Bryn Pernot

What's Holding Us Back? Rethinking Outcomes-Based Evaluation in New Contents
Jill Stein, Shelly Valdez, Martin Storksdieck

This working group seeks to address and re-think the field’s embeddedness in outcomes-based evaluation models that focus on individuals, particularly as we increasingly engage with communities that may be less outcomes-based. The goal is to gather insights toward a white paper on thinking outside the “outcomes-based box.”

Situational Interest: What is It and How Do We Measure It?
Marcie Benne, Josh Gutwill, Monae Verbeke

Situational interest, or the positive emotional engagement that motivates attention and learning, is fundamental to experiences in museums and other informal/free-choice learning environments. Join this active discussion to review recent approaches to studying situational interest and explore challenges, successes, and future directions for the field.

Risk and Reward: Taking Chances with Reliability
Renae Youngs, Elee Wood, Susan Foutz, Kirsti Lekies

When do you use an existing, validated evaluation instrument, a modified instrument or one that's "home grown"? Each affects the reliability and validity of findings - for good or ill. Join a lively discussion of when, how, and why to bend or break social science "rules" about instrument reliability and validity.

The Social Side of STEM: Evaluating SciCafes and Public Forums
Jennifer Borland, Elizabeth Kunz Kollmann, Chris Cadenhead, Kathryn Rende

This presentation explores SciCafes and other live, informal events where public audiences hear from and interact with scientists and have socialization opportunities. Panelists representing multiple institutions will discuss evaluative methodologies and impacts of these programs on participants' understanding of science, motivations to learn more, and resulting behavioral changes.

Concurrent Sessions | 1:15pm - 2:30pm

PAPERS: Family Learning

Children and Adults' Interaction at a Brazilian Science Exhibition
Rosicler Neves, Luisa Massarani

Parent Roles During Mobile-Based Educational Experience on a Family Hike
Lucy McClain

Finding the Gap: Museums in Service to Young Multi-Lingual Learners
Rita Deedrick

Writers Block: When Reporting Gets in the Way of Use
Ryan Auster, Lauren Wilson, Sarah Lee

Written reports are the culminating aspect of many projects, but evidence suggests that reports may lack quality and are often under-utilized. How do we, as researchers and evaluators, produce high-quality, useful reports efficiently? This working session will allow attendees to have critical conversations around methods for improving our reporting strategies.

Elevating the Value of Social Science in Science-Based Institutions
Nette Pletcher, Lisa Trahan, Kathy Kiser, Mary Jackson

Explore ways to convey the importance of the social sciences to hard-science colleagues. Gain strategies for communicating the rigor and impact of evaluation with those who may not fully appreciate your work. Case studies drawn from science-based research institutions, but applicable also to history and art museum professionals.

Accessing Evaluation: Creating Evaluation Instruments for Visitors with Disabilities
Mary Ann Wojton, Deborah Wasserman

In order to give all participants a voice, evaluation instruments need to be accessible to all, including individuals who have physical, cognitive or developmental disabilities. Evaluators who have worked with programming for people with special needs will share outcome measurement instruments that are accessible for the physically, cognitively and developmentally disabled.

Collaboration as Process, Collaboration as Product—Navigating and Embracing Complexity
Jim Kisiel, Cecilia Garibay, Marjorie Bequette, Rich Pennington

Although messy and unpredictable, partnerships are critical for developing inclusive practices and building institutional capacity. Here, we describe several cases involving community collaborations, viewed as either a means (data-collection method) or end (project outcome), leading to an audience-driven discussion involving shifting evaluator roles, collaboration frameworks and lessons learned.

Roundtables | 3:00pm - 3:45pm

The Polls Were Wrong: Confronting Methodological Pitfalls
Amanda Krantz, Jen Benoit-Bryan, Jennifer Borland

As evidenced in 2016's US presidential election, even the most experienced pollsters can be wrong. Methodological pitfalls are present in our field too. In this discussion-based session, we will identify challenges to commonly used visitor studies methodologies, propose solutions, and document them for the field to learn from.

You Keep Using that Word: Thematic Analysis of VSA Voices
Jeremy Foutz

Using latent Dirichlet allocation (LDA) and critical discourse analysis methods, this roundtable examines the language and potentially latent themes we use in wrestling with VSA's diverse topics and concerns. This study is exploratory and a productive way to explore what VSA is "about" by analyzing documents and conference presentation texts.

Teachers as Reviewers, Informants, Guests, and Guides
Alice Anderson

In designing and evaluating school programs we need to effectively and efficiently gather information about what teachers value, need and want. We will share strategies and challenges for listening to teachers and administrators and how we use this data to inform decision making and advocacy within our museums.

Shared Language for Aligning the Visitor Experience and Design Intent
Lindsay Maldonado, Kris Nesbitt

Creating and assessing the visitor experience requires a thoughtful understanding of your audience, and the ability to reflect on your own assumptions about how visitors experience your institution. This session will present how evaluators and designers can collaborate to unpack differences between the visitor experience and design intent.

Making Observations: Identifying Evidence of Learning in Makerspaces
Peter Wardrip, Lisa Brahms

Making Observations is a project of the Children's Museum of Pittsburgh and funded by the Institute of Museum and Library Services. The project goal is to develop a suite of valid and reliable evaluation tools and approaches for assessing learning through making by practitioners, evaluators and researchers in museums and libraries.

Poster Session | 4:00pm - 5:15pm

Saturday, July 22, 2017

Closing Plenary & Breakfast | 8:00am - 9:30am

Invited Speaker: Sathya Gopalakrishnan, STEAM Factory and Ohio State University

Sathya Gopalakrishnan is an Assistant Professor of Environmental Economics in the Department of Agricultural, Environmental and Development Economics at The Ohio State University. She is also a founding member and the Director of Evaluation and Assessment at The STEAM Factory. Based at The Ohio State University, The STEAM Factory provides a space where researchers from all disciplines can broaden their perspectives, share resources, spark creative research ideas, and form collaborations across areas of common interest.

Concurrent Sessions | 9:45am - 11:00am

Investigating Pathways to STEM Identity in Free-Choice Learning Environments
John Falk, Lynn Dierking, Smirla Ramos-Montañez, Ivel Gontan, Heidi Carlone

Although many scholars have embraced identity as an important focus for investigation, the concept remains challenging to operationalize and study. Presenters will provide an overview of research on STEM self/identity and highlight three recent studies with adults and youth from traditionally underserved communities participating in free-choice learning programs.

Diverse Worldviews in Museums: How Evaluation Can Open New Pathways
Jill Stein, Shelly Valdez, Nancy Maryboy

This panel shares results from multiple federally-funded projects that bridge Indigenous knowledge and western science, and reveals how science and tribal museum visitors are more open to exploring diverse worldviews than assumed. Participants will discuss how evaluation can highlight differences and similarities in institutional and visitor openness to diverse worldviews.

Cyberlab Instrumentation for Innovative Learning Research in Multiple Scenarios
Shawn Rowe, Susan Rowe, Mark Farley

The Cyberlab learning research laboratory employs new technology to transform exhibits into research platforms and to provide remote data collection tools for researchers around the world. This panel will invite participants to discuss our new paradigm for informal learning research and how it can contribute to larger contemporary research agendas.

Leaving the Temple on a Hill: Evaluating Art Museum Community Engagement
Elizabeth Bolander, Caitlin Tracey-Miller, Laura Weiss

Three art museums will share different approaches to evaluating community engagement. This research acts as a catalyst for institutions to consider how they connect with their communities. Panelists will reflect on the role evaluation can play in bringing a greater voice to non-visitors, partner organizations, and other community members.

Dressing Up (In) the Exhibit: Impact of Costumes in Museums
Claire Thoma Emmons, Merilee Mostov, Valerie Grabski, Elee Wood

Costumes in the exhibit: what is their intended purpose? Who actually uses them? Can they increase stay time? Will they promote pretend play? Will they support social interaction? Panelists from three different museums will present data concerning costume use in various exhibit environments and discuss the implications with attendees.

Concurrent Sessions | 11:15am - 12:30pm

PAPERS: Findings from Zoos and Nature

The Role of Empathy and Curiosity in Facilitating Social Change
Jerry Luebke

Evaluating NatureStart Network: How Nature Play Partnerships Benefit Urban Families
Jennifer Matiasek

Polar Opposites: Video Attraction Rate at an Animal Exhibit
Erin Tate

Making Connections: Facilitating Learning through Making
Lisa Brahms, Peter Wardrip

Making Connections is a professional learning card game designed to encourage critical discussion about how best to facilitate maker-based learning. Derived from related empirical studies in MAKESHOP at Children's Museum of Pittsburgh, the game cards represent three areas of consideration: the learners, the learning objectives, and the facilitation strategies.

Data Collection with Sensitive Populations
Aaron Price, Gloria Segovia, Bryn Pernot, Fran Mast

This panel will discuss collecting data from non-dominant populations using methods based on both latest scholarship and the personal experiences of researchers at three informal learning institutions. Topics include quantitative and qualitative methodology, accessibility and recruitment.

Sharing is C(omp)aring: The Value of Cross-Institutional Data
Alexander Lussenhop, Joe Heimlich, Elisa Israel

Join members of the Governing Body of COVES, the Collaboration for Ongoing Visitor Experience Studies, as we discuss the importance (and challenges) of collaboration, share date from initial analysis of data from our ongoing multi-institutional study of science centers, and turn to you for questions and suggestions for future direction.

To register for the 2017 VSA Conference, click HERE.