During a recent event for kids at New York’s Metropolitan Museum of Art, visitors put on virtual-reality goggles and looked at Jackson Pollock’s “Autumn Rhythm (Number 30).” As they gazed through their 3-D headsets, splatters of black and white paint appeared to break free from the 1950 canvas and float past their eyes. Children and adults tried to touch the illusion, waving at the images in the air.

Museums are flirting with change that may be more revolutionary than at any other point in their history. The forces rocking the technology world—cheaper screens, miniaturized mechanics and increased computing power—are prompting a rich period of experimentation in exhibit design. For museums, such advancements could attract diverse visitors, lure young people and change the way audiences learn about art, science and nature.

“There’s a tremendous amount of innovation in museums right now—it’s a really exciting time,” said Kurt Haunfelner, vice president of exhibits and collections at the Museum of Science and Industry, Chicago. Museums are “aware that the expectations of their audiences are changing fairly dramatically and I think they are committed to experiment and prototype different approaches to engaging an audience.”

Around the globe, major institutions are trying out new digital tools. During the summer, the Natural History Museum in London unveiled a virtual-reality film that recreates ocean creatures from 500 million years ago. This year, the Illinois Holocaust Museum & Education Center began demonstrating a life-size interactive video of a Holocaust survivor that it hopes one day to display in three dimensions like a hologram. The Met, which created a media lab about three years ago to investigate how technology can change visits to its galleries and test other ideas, is exploring a colorful way to present its famed Temple of Dendur.

“The MediaLab is looking at what the museum will look like in 20 to 25 years,” said lab manager Marco CastroCosio.

While aimed at boosting crowds, the new technology could blur the lines between education and entertainment. As institutions focus on the thrill of new displays, some observers caution that the art and artifacts could get lost in the commotion. “It’s important that museums be responsive to audiences, but it can tip easily into making it all about the audience and maybe not pushing the audience into something outside their own experiences,” said Elizabeth Rodini, director of the museums and society program at Johns Hopkins University.

The digital onslaught also raises questions about who is controlling the content—the museums or the experts advising them. It’s now routine to find museums working with tech-savvy design firms whose portfolios include theme-park rides, live shows and corporate events. Museum professionals who create theatrical exhibits are getting noticed—people like Mr. Haunfelner, a 10-year veteran of Walt Disney Imagineering whose immersive shows have helped raise the Chicago museum’s profile.

“When we talk to museum guys, they’re very serious, they say, ‘We don’t want that goofy theme-park stuff,’” said Adam Bezark, creative director at the Bezark Company, a Los Angeles design firm. “And so we say, ‘OK, we’ll bring in our serious people.’ And it’s all the same people. A great designer is a great designer.”

Museums are not only interested in technology but in storytelling—pacing, suspense, the big reveal. Over the last decade, professionals with experience in the entertainment world have collaborated more with cultural institutions, receiving fewer “icy stares of death” from museum directors as they pitch their ideas, said Mr. Bezark, who described growing respect and admiration on both sides of the table. Mr. Bezark, whose clients include Disney and Universal theme parks, worked on the story concept and presentation of a film that plays at the National WWII Museum in New Orleans.

In some cases, museums are allowing multimedia experts to take the lead. A movie that will play at the American Revolution Museum at Yorktown, a Virginia museum set to open next year, was conceived, written and produced not by the museum but by the design firm Cortina Productions. Museum curators consulted throughout the process, said Heather Hower, the museum’s project manager for media.

The movie features costumed actors and special effects in a newly built 4D theater with piped-in gunpowder smells and seats that jiggle whenever cannons are fired. The 8-minute film, which cost $775,000 to produce, is “a wonderful, technology-driven presentation” that will reach visitors in a way that a documentary would not, said Ms. Hower.

Alan Maskin, whose Seattle design firm Olson Kundig works with cultural venues, said augmented reality has “the greatest potential for changing museums” because it allows visitors to tumble down dozens of rabbit holes once inside an exhibit. Visitors could use virtual-reality headsets or other tools to access extra digital material. That much freedom could mean curators have less power to shape their message, he said, adding: “The author or curator loses some control of the experience when the visitors are given more choice and their experiences become more self-directed.”

In the Metropolitan Museum of Art’s MediaLab, a small team is tinkering with everything from augmented reality to videogames. “We’re working on the future of culture,” said Sree Sreenivasan, the Met’s chief digital officer whose team is responsible for the interactive Pollock display using Oculus Rift, a virtual-reality system. “It’d be fair to say we’re looking for ways to learn from Silicon Valley.” Ideas tested by the lab’s design, media and tech interns include an augmented-reality project for the Temple of Dendur. The experiment would illuminate the Aeolian sandstone structure in what are believed to be its original vibrant blues, greens and yellows using projection mapping, a digital tool that imposes rich images on an object’s surface. If adopted, the temple could flash into color periodically for all visitors to see, or the colors could appear on individual smartphones when visitors scan their screens over its surface.

During the summer, Director Thomas Campbell traveled to Silicon Valley to visit the offices of Oculus, Facebook and Instagram. Mr. Campbell posted on Instagram a blurry picture of Mr. Sreenivasan trying out an Oculus Rift headset, “about to be eaten” by a Tyrannosaurus Rex in a virtual-reality demonstration. “Remarkable to think that within 10 years virtual reality will be a big part of mainstream entertainment,” Mr. Campbell wrote.

Science, history and children’s museums have long featured multimedia exhibits. Today, the bar is higher for those museums whose shows are judged partly by the attention they generate on social media. Art museums, once seen as too highbrow for interactive exhibits, are starting to enter this territory, too. Last year, the de Young museum in San Francisco became one of the first art museums to integrate wearable technology when it unveiled a Google Glass tour for its Keith Haring show.

Smaller museums can serve as test sites for risky, pioneering ideas. This year, the Illinois Holocaust Museum & Education Center began showing a video of Pinchas Gutter, a Holocaust survivor who answered roughly 1,200 questions about his life and the war in more than 20 hours of recorded interviews. Today, voice-recognition software picks out Mr. Gutter’s responses based on key words in visitor queries (he even has a response if someone asks his favorite color). The Toronto octogenarian sometimes cries, grows angry, laughs, sings and expresses joy. Even in two dimensions, the life-size video appears realistic enough that visitors say hello to Mr. Gutter and apologize if they cough while he is talking, said Amanda Friedeman, the museum’s survivor community liaison.

The project, called “New Dimensions in Testimony,” is a collaboration between the University of Southern California’s Shoah Foundation and the USC Institute for Creative Technologies, with the aim of making Holocaust survivors one day appear similar to holograms in full three-dimensional form. Over the next few months, the Shoah Foundation plans to film about a dozen Holocaust survivors inside what looks like an 18-foot-tall soccer ball—a set with a wraparound green screen, 6,600 LED lights and roughly 100 cameras. Survivors sit under the lights for a week answering questions, at a cost of $500,000 per interview, paid for mostly with private donations. Stephen Smith, the foundation’s executive director, said projections that look like holograms and don’t require 3-D glasses are likely in the next two years.

Exhibit-design companies are touting the next big thing at trade shows attended by museum professionals. At one such event in Atlanta this year, Group Delphi was marketing the “Wonder Wall,” a series of screens that appear blank except to those wearing polarized glasses. “It’s sort of like, you can’t do this at home,” said Group Delphi’s executive producer of digital content, Shannon Densmore. The product costs roughly $45,000.

Investments in technology can backfire. Anything too adventurous runs the risk of becoming irrelevant, especially in museums that take years to install new designs. For example, QR codes, the black-and-white images that viewers can scan with their phones to access extra material beyond wall text, “now look somewhat dated” compared with more sophisticated geo-positioning devices found in many museums, said Leonard Garfield, executive director of the Museum of History & Industry in Seattle.

At the Broad, the newly opened contemporary art museum in Los Angeles, technology is deployed sparingly in public spaces—most visibly with mobile devices that assistants and gallery guides carry to speed admission and answer questions. “What you don’t want to have is a lot of technology pasted into an existing museum approach,” said Joanne Heyler, the Broad’s founding director. “It might be easy to get seduced into how beautiful some technologies are, but you always have to put yourself in visitors’ shoes and think about why they are coming to a museum.”

But high-tech displays can capture the imagination, too. London’s Natural History Museum scored a popular hit with “First Life,” a virtual-reality film starring hallucinogenic ocean creatures brought to audiences in color using museum-backed research. Visitors wear headsets inside a 65-seat theater and experience 360-degree views of prehistoric sea life swimming around them. Producer Anthony Geffen said the educational film goes beyond the “dusty little fossils” also featured in the exhibit. Mr. Geffen, who made the film with British broadcaster David Attenborough, said seven American museums plan to show it.

Still, any movie that claims to know the color of ancient sea life necessarily takes some educated guesses, said Harvard University natural history professor Andrew Knoll. While fossils provide reliable information about the size, shape and function of these ancient animals, other details are educated guesses based on the look of a creature’s distant relatives. He watched an online trailer for “First Life” and stopped at the Opabinia, an extinct relative of arthropods. Though the creature accurately appears with five eyes atop its head, Mr. Knoll balked at its striking colors. “Do we know whether it had blue spots on it and a green head? No, of course not,” he said.

Mr. Geffen said scientists who worked on the film used the best available information to produce the prehistoric life. “Nobody is saying we know for sure every single color,” he said. “We’d never pretend that everything is absolutely accurate because the experts themselves say that they have just worked on what they’ve got to go with.” Emily Smith, the museum’s head of audience development, said: “The inferences made in ‘First Life’ are consistent with those made in other areas of the museum that address extinct creatures, such as our dinosaur gallery. All depictions of extinct creatures in the museum are reconstructions based on the best understanding of scientists at the time.”

The museum warns adults with heart conditions, psychiatric disorders and other health issues to consult their doctors before buying the nearly $10 ticket. Children under 13 are not admitted. But such warnings don’t appear to be scaring many people away: The museum has extended the show’s run into January.