Computational imaging starts from a simple idea: design the camera around the information you need, not just around the picture a person expects to see. Hiroyuki Kubo, an associate professor at Chiba University’s Graduate School of Informatics, puts it this way: “If we change the way we capture data, the potential of Artificial Intelligence (AI) expands dramatically.” His lab uses ordinary projectors and cameras to reveal blood vessels beneath the skin and the direction of flowing water, and it builds AI that helps artists colour anime.
The interview, published by Chiba University through EurekAlert! on 27 September 2026 and republished by Tech Xplore, covers two research lines that look unrelated. They turn out to share one philosophy. In the first, physics decides what the camera records before any AI sees the data. In the second, AI proposes and a human decides, because an animation studio cannot tolerate even small errors.
This article explains both, traces the published papers behind the interview, and sets out what the approach means for businesses building computer vision systems of their own.
Table of contents
- What Computational Imaging Changes About the Camera
- Seeing Blood Vessels Beneath the Skin With Computational Imaging
- Computational Imaging Makes Water Flow Visible With Nanofibres
- Why Fully Automatic Anime Colouring Fell Short
- AI Proposes, the Artist Chooses
- GENIAC and the Future of Japan’s Anime Industry
- From Physics to Pixels: Kubo’s Route Into Computational Imaging
- How Kubo Chooses Research Problems
- Timeline of the Computational Imaging and Colouring Papers
- What Computational Imaging Means for Businesses
- Computational Imaging FAQs
- References
What Computational Imaging Changes About the Camera
Most AI imaging research starts after the shutter has closed. A photograph is taken, and software tries to extract as much meaning from it as it can. Kubo works on the other side of that line. “I focus on the capture process itself,” he says, “rethinking how cameras, lighting, and lens settings are used” to obtain visual information “that cannot be perceived by the human eye or obtained through conventional imaging methods.”
That is the core of computational imaging: the optics, the light source and the algorithm are designed together, so the recorded data already contains the signal the analysis needs. The field has its own flagship venue, the IEEE International Conference on Computational Photography (ICCP), where Kubo and his colleagues presented one of the key papers behind this interview in 2018.
The milk and liquid soap problem
Kubo’s favourite illustration is a glass of white liquid. Show a photograph of it to an AI system and “it will most likely be identified as milk. However, the liquid could just be liquid soap.” A standard photograph “records only the light reflected from an object’s surface”, so it says very little about what the object is made of or what is inside it.
The two liquids differ in how light travels and scatters inside them. “By selectively capturing light that penetrates the liquid and returns to the camera, I can visualize these internal differences,” Kubo explains. Train a model on that kind of computational imaging data and it may be able to separate materials that look identical to people.
Why better data beats a bigger model
The lesson for anyone working with AI is uncomfortable but useful. When two things produce the same pixels, no amount of model capacity can tell them apart reliably, because the distinguishing information was never recorded. Computational imaging fixes the input rather than the model. It is the imaging equivalent of the data-quality rule that decides most machine learning projects.
| Factor | Conventional photograph | Computational imaging capture |
|---|---|---|
| What is recorded | Light reflected from the surface | Selected light paths, including light that entered the material |
| Hardware | A standard camera with ambient or flash light | An ordinary projector and camera, synchronised and controlled |
| Milk or liquid soap | Looks identical, so AI guesses milk | Different internal scattering becomes visible |
| Veins under the skin | Hidden by strong surface reflection | Visible in real time, without radiation |
| Direction of water flow | Cannot be seen in a still image | Visible as a polarisation pattern from aligned fibres |
| Where the effort goes | Mostly software, after capture | Split between optics, capture timing and software |
Seeing Blood Vessels Beneath the Skin With Computational Imaging
The best-known result of this computational imaging work is a live view of veins. “When a standard photograph of the skin is taken, the light reflected from its surface is too strong to reveal the blood vessels underneath,” Kubo says. His system controls the projector and the camera precisely and introduces “a slight spatial offset between the illumination and imaging positions”. The camera then records light that has travelled through the tissue and interacted with the vessels.
How the offset works
The published version of this computational imaging technique dates back to 2018. In a paper at ICCP that year, Kubo, then at the Nara Institute of Science and Technology (NAIST), worked with Suren Jayasuriya, Takafumi Iwaguchi, Takuya Funatomi, Yasuhiro Mukaigawa and Srinivasa Narasimhan of Carnegie Mellon University. They described a rectified projector-camera system in which they “vary the offset between projector and camera rows (implemented as synchronization delay) as well as the exposure of each camera row.”
In plain terms, the camera reads one row of pixels at a time, and the projector lights one row at a time. Shift the two out of step and the camera only sees light that has wandered sideways through the scene before returning, which is the light that has passed beneath the surface. The resulting “delay-exposure stack of images” can “disambiguate subsurface scattering, diffuse and specular interreflections, and distinguish materials according to their subsurface scattering properties.”
From paper to possible diagnosis
An extended version of the computational imaging method appeared in IEEE Transactions on Visualization and Computer Graphics in April 2021, where the authors showed “the utility of indirect imaging for capturing and analyzing the hidden structure of veins in human skin.” In the new interview, Kubo stresses the practical side. The computational imaging system can “visualize blood vessels in real time without exposing patients to radiation,” and “because the system is compact and relatively inexpensive, it has potential applications not only in clinical settings but also in home healthcare and disaster-response environments.”
The AI angle comes next. “By training AI on the images obtained through this method, it may be possible to support the diagnosis of vascular conditions such as varicose veins.” That is a research direction, not a clinical claim, and any diagnostic use of computational imaging would need the usual trials and regulatory approval first.
Computational Imaging Makes Water Flow Visible With Nanofibres
The second computational imaging example in the interview is less obvious and more industrial. “From a conventional photograph, it is impossible to determine whether water is flowing or in which direction it is moving,” Kubo notes. His computational imaging answer was to add something to the water that changes how light behaves.
Cellulose nanofibres and polarised light
Kubo disperses “extremely fine fibers—cellulose nanofibers—into the water.” In still water the fibres point in random directions, so the polarisation of the light they scatter varies too. When the water flows in one direction, “the fibers align with the flow,” and that alignment “produces a characteristic polarization pattern.” A camera that records polarisation, the direction in which light waves vibrate, can then map the flow.
The nanofibre computational imaging work was presented as a poster at ACM SIGGRAPH 2024 under the title “Visualization of Flow Direction using Polarization Angle Changes of Cellulose Nanofiber Suspension”, by Ryusei Okamoto, Shogo Yamashita, Takuya Kato and Kubo. The listed affiliations are Chiba University and ExaWizards Inc., a Japanese AI company, which fits the interview’s description of “a collaborative project with a company that uses AI.”
Where flow imaging could be used
Kubo lists three uses for combining these computational imaging results with AI: “more efficient maintenance and management of infrastructure such as bridges, the design of fuel-efficient ships, and the analysis of swimming techniques.” All three involve water moving around a structure or a body, where seeing the flow directly is usually expensive or impossible. The table sets his computational imaging applications side by side.
| Application | What is normally invisible | How it is captured | Possible AI use |
|---|---|---|---|
| Veins beneath the skin | Vessels hidden by surface glare | Offset projector and camera rows isolate light that passed through tissue | Supporting diagnosis of conditions such as varicose veins |
| Look-alike liquids | Internal structure of milk and liquid soap | Selective capture of light that entered the liquid | Telling materials apart that look identical |
| Water flow | Direction and pattern of moving water | Cellulose nanofibres align with the flow; the camera records polarisation | Bridge maintenance, ship design, swimming analysis |
| Vehicles in fog | Objects behind scattering droplets | Named as a target in the interview; no method details given | Safer driving and monitoring in poor visibility |
Why Fully Automatic Anime Colouring Fell Short
The second half of the interview moves from physics to production. “Since around 2018, I have been conducting joint research with an anime production company,” Kubo says, with the goal of “improving the efficiency of the anime production process using AI.” Of all the stages, he singles out “the process of coloring line drawings” as the one that “requires a significant amount of labor.”
Here computational imaging gives way to a different question: how much of a creative process should a machine own?
Training on tens of thousands of pairs
His first attempt was the obvious one: let the AI apply the colours automatically. “I prepared thousands—sometimes tens of thousands—of pairs of line drawings and their fully colored versions, and used them to train the AI.” The published record matches. A SIGGRAPH Asia 2018 poster by Sophie Ramassamy, Kubo and colleagues described “pre- and post-processes for automatic colorization using a fully convolutional network.”
It worked, but not well enough. “We soon realized that achieving 100 percent accuracy would be extremely difficult,” Kubo says.
Why one per cent is too much
The reason is the tolerance of the medium. “In anime production, even a one-percent coloring error can be critical.” A wrong colour on a character’s hair or uniform is visible on screen, and hunting down scattered errors across many frames can erase much of the time the automation saved. An automatic system that is right 99% of the time is therefore not 99% useful. That realisation, Kubo says, “led me to rethink my research approach.”
AI Proposes, the Artist Chooses
“After considerable trial and error, I developed a system in which AI and humans collaborate in the coloring process,” Kubo explains. “In this system, the AI first proposes several candidate colors, and a human artist then selects the appropriate one.” The approach “can significantly reduce the amount of work required while maintaining a high level of accuracy in coloring.” It also changes what the training data is for: instead of dictating one colour, the model ranks the likely ones.
The papers behind the approach
The published work shows how the idea developed. A SIGGRAPH 2019 poster on “graph matching based anime colorization with multiple references” was co-authored by Akinobu Maejima of IMAGICA GROUP with Kubo and NAIST colleagues. At SIGGRAPH 2020, a talk titled “Confidence-aware Practical Anime-style Colorization” listed co-authors from OLM Digital, part of IMAGICA GROUP, and from NAIST.
That 2020 talk contains the seed of the current design. Its “key idea is the strategic withdrawal which reflects the prediction confidence,” meaning the system only colours the regions it is sure about and leaves the rest to people. The authors studied the balance between confidence, accuracy and the number of automatically coloured regions “to maximize the efficiency of the colorization process including both automatic prediction and manual correction.”
Later work covered few-shot learning for character colouring at SIGGRAPH Asia 2021, and a 2022 paper at Nicograph International evaluated a “semi-automatic colorization pipeline for anime characters” in real production, by which time Chiba University appears among the affiliations. A SIGGRAPH Asia 2025 poster proposed a “fisheye patch-based automatic colorization method for anime line-drawings.”
| Factor | Fully automatic colouring | AI proposes, artist chooses |
|---|---|---|
| Who decides each colour | The model | The artist, from AI candidates |
| Training data | Thousands to tens of thousands of line drawing and colour pairs | The same kind of data, used to rank candidate colours |
| Accuracy target | 100%, which proved “extremely difficult” | High accuracy, secured by human selection |
| Effect of a 1% error | Critical, and the errors still have to be found | Caught at the moment of selection |
| Artist’s role | Checks and repairs the output | Makes quick choices among suggestions |
| Fit for broadcast work | Poor on its own | Designed and evaluated for production use |
A model for human and AI teamwork
Kubo sees this as more than a workflow tweak. “I believe this kind of collaboration represents one possible model for the relationship between humans and AI in the next generation.” It is the same pattern many businesses now use for intelligent automation: let the system search and draft, and keep a person on the decision that carries the risk.
GENIAC and the Future of Japan's Anime Industry
Since December 2024, Kubo has also taken part in the “Generative AI Accelerator Challenge: GENIAC,” which he describes as an initiative “implemented by Japan’s Ministry of Economy, Trade and Industry, and the New Energy and Industrial Technology Development Organization (NEDO) to strengthen the country’s capabilities in generative AI development.” He hopes it will “further accelerate” his research, with the aim of “supporting the future of Japan’s anime industry.”
What GENIAC provides
METI and NEDO launched GENIAC in February 2024. According to METI, it “provides support for the provision of computing resources, assists in demonstration projects for data utilization, organizes matching events, and facilitates collaboration with global tech companies.” For a university lab, access to that kind of compute is often the difference between a promising poster and a model that can be tested at production scale.
Why anime is a sensible target
Anime is labour-intensive, highly stylised and extremely consistent within a series, which makes it a good fit for assistance that learns a show’s colour design. It is also a sector where artists are wary of automation that replaces them. A system that proposes and lets the artist choose sidesteps much of that tension, because the creative decision stays with a person.
From Physics to Pixels: Kubo's Route Into Computational Imaging
Kubo did not start in computer science. “When I first entered university, I was interested in areas of physics such as cosmology and elementary particle physics,” he says. Wanting to work on problems “closer to everyday life”, he joined a lab working on facial image analysis in his third undergraduate year, and later earned a doctorate in computer graphics using software-based approaches.
Camera maker, NAIST and Carnegie Mellon
After the doctorate he “worked for a camera manufacturer,” which exposed him to “the technical expertise behind the photographic process” and sparked an interest in hardware. At NAIST and Carnegie Mellon University he “discovered the excitement of research that combines software and hardware.” That combination is exactly what defines computational imaging.
A rare mix of software and hardware
He thinks that mix, the heart of computational imaging, is unusual. “Most researchers in image and information processing specialize primarily in software, and relatively few—probably less than ten percent—also work on hardware, as I do.” He credits his physics background for making it easier to move into optics “without hesitation.”
Taken at face value, Kubo’s estimate means at least nine in ten researchers in his field never touch the optics, which is why computational imaging groups remain small.
How Kubo Chooses Research Problems
Kubo’s advice on picking topics is one of the most quotable parts of the interview. “One thing I try to focus on is identifying problems that few people have noticed or attempted to tackle.” Crowded fields, he says, turn into a contest “like runners sprinting in a 100-meter race,” where everyone competes on speed under the same rules.
Setting your own rules
“If you choose a problem that few others have explored, you are free to set your own rules and approach it creatively,” he says. He compares it to “an unusual challenge—like trying to juggle several very different things while sprinting at full speed.” Computational imaging suits that approach, because combining optics, electronics and machine learning is hard enough that few groups attempt all three.
Advice for students
He offers two pieces of advice for early-career researchers. First, “engage actively with the wider community” by attending events, conferences and research meetings, because “unexpected connections lead to valuable support.” Second, practise explaining your research “in a way that even your parents or siblings can understand,” without jargon or formulas. The milk-and-soap example is that advice in action.
Timeline of the Computational Imaging and Colouring Papers
The papers behind the interview span eight years, from 2018 to 2025. The table lists the ones cited in this article, and the chart below it counts them by research strand.
| Date | Venue | Work | Strand |
|---|---|---|---|
| May 2018 | IEEE ICCP | Acquiring and characterizing plane-to-ray indirect light transport | Light transport and veins |
| Dec 2018 | SIGGRAPH Asia Posters | Pre- and post-processes for automatic colorization using a fully convolutional network | Anime colouring |
| Jul 2019 | SIGGRAPH Posters | Graph matching based anime colorization with multiple references | Anime colouring |
| Aug 2020 | SIGGRAPH Talks | Confidence-aware practical anime-style colorization | Anime colouring |
| Apr 2021 | IEEE TVCG | Programmable non-epipolar indirect light transport: capture and analysis | Light transport and veins |
| Dec 2021 | SIGGRAPH Asia Technical Communications | Anime character colorization using few-shot learning | Anime colouring |
| Jun 2022 | Nicograph International | Semi-automatic colorization pipeline for anime characters and its evaluation in production | Anime colouring |
| Jul 2024 | SIGGRAPH Posters | Visualization of flow direction using polarization angle changes of cellulose nanofiber suspension | Water flow |
| Dec 2025 | SIGGRAPH Asia Posters | Fisheye patch-based automatic colorization method for anime line-drawings | Anime colouring |
Six of the nine papers above deal with colouring, so the anime work is the larger body of published output, even though the camera work gets the headline.
What Computational Imaging Means for Businesses
Most organisations will never build a projector-camera rig. The principle behind computational imaging still applies to any business deploying image-based AI, from factory inspection to document scanning.
Fix the capture before the model
If an inspection model keeps confusing two defects, the answer may be a different light, angle, wavelength or polarising filter rather than a larger model or more labels. Computational imaging makes that trade explicit: a cheap change at capture can remove an ambiguity that no amount of training can resolve.
Keep people on high-cost decisions
Computational imaging is only half of Kubo’s lesson. The anime colouring system shows where automation should stop. Where an error is expensive and hard to spot, have the AI narrow the options and a person confirm. The same pattern works for invoice coding, document review and quality control, and it is how our ML model development projects usually structure review steps.
Budget for hardware skills
If fewer than one in ten image-processing researchers work on hardware, as Kubo estimates, people who can design a computational imaging system are scarce. Projects that need custom optics should plan for that skill early, through a university partnership or a specialist supplier. Our AI strategy work often starts by asking whether data capture, not modelling, is the real bottleneck.
Watch sensor-level AI too
Computational imaging is one of several moves that push intelligence closer to the sensor. We recently covered a stacked AI chip that uses fast and slow layers to recognise motion, and another university Q&A on reverse problem generation in computational AI mathematics. The common thread is that better inputs, not just bigger models, drive the next gains.
Computational Imaging FAQs
What is computational imaging?
Computational imaging designs the optics, lighting and software of an imaging system together, so the camera records the information an algorithm needs rather than only a conventional picture. Kubo’s projector-camera systems are one example.
How does computational imaging reveal blood vessels under the skin?
He synchronises an ordinary projector and camera with a slight offset between the row being lit and the row being recorded, so the camera captures light that travelled through the tissue. The method works in real time and uses no radiation.
Does the anime system colour frames automatically?
Not entirely. The AI proposes several candidate colours for each region and an artist chooses one. Kubo moved to this design because even a 1% error rate is too high for broadcast production.
What is GENIAC?
GENIAC, the Generative AI Accelerator Challenge, is a programme launched in February 2024 by Japan’s Ministry of Economy, Trade and Industry and NEDO. It supports generative AI development in Japan, mainly by providing access to computing resources. Kubo has taken part since December 2024.
Where was the computational imaging interview published?
Chiba University released it through EurekAlert! on 27 September 2026, and Tech Xplore republished it.
References
Seeing the unseen, coloring anime: AI and physics shaping the future of imaging (EurekAlert!)
Seeing the unseen, coloring anime (Tech Xplore)
Acquiring and Characterizing Plane-to-Ray Indirect Light Transport (NAIST project page)
Confidence-aware Practical Anime-style Colorization (ACM SIGGRAPH History Archive)
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.
