By Lin Yu-Ching  

Photographs by Janet Yeh 

"In 2006, the first year I took up the post as the PI of the Research & Development of Technology Division in the National Digital Archives Program (NDAP), what I complain about most often at home is: 'why the heck they come up with this sort of program? What on earth are they doing? Why am I so bloody tired?'" The speaker recounts a story that seems to be as remote as what happened during the Tianbao Era of Tang Dynasty, though not without entertaining quips – he is Mark Liao, distinguished research fellow at the Institute of Information Science, Academia Sinica.

Titled "Multimedia Research and Digital Archives," his talk brings forward what he is very good at: manipulating multimedia signals. More than a decade ago, facing masses of old photographs and impaired images cropping up in film tapes covered with mold, stains or breaches, his job has been to develop a technology for digital visual repair to deal with the casualties resulted from moist and age, in so far as what remains of a vintage picture can be restored to deceiving perfection.

In digital repair, fixing black-and-white images is deemed tougher than fixing the colored ones – this led Mark to reassess the competences of his lab, not only apropos of how to fulfil what was expected of his team but also of how to open up spaces for further scholarship. With the paucity of this kind of technology in the world a few years after the second millennium, Mark led his team to sally into "something worth trying a bit" – video inpainting.

"The question concerning video inpainting is that whether I can get rid of what I don't want or what's broken in the images and then fill up the blank so that nothing seems to have happened on the face," explains Mark. "What happens with the 'in-' with 'inpainting' is to apply what is still good to damaged regions." A work of a master-restorer should therefore pass the eyes of all, like Frank Abagnale starred by Leonardo DiCaprio in Catch Me If You Can – he pays his way with bamboozling fake checks and gets absorbed into FBI as top inspector of forgery after his arrest. The idea of forgery, put in the other way round, inspires the methodology of Mark's team to carry out the national project of digital archives.

And it is in the same gist of cunning and trial and error that Mark has blazed a trail through what he dubs as "mandatory research," something that is assigned from top to down and one can hardly walk away from without failing one's duty. His team manages to tackle cases ranging from global motion that is relatively easier to fix, to local motion (e.g. moving figures or objects in a motion picture), and then to a specific spot on a moving object that is genuinely harder than the preceding cases, because one needs to accurately restore every phase of the object's movement in order to get the spot right. The task initially catered to the needs of National Archives Administration, National Development Council, but it has also grown into a project producing a good many papers, one of which was accepted by IEEE Transactions on Multimedia, pushing Mark "on the verge of joyous tears because that was terrific!" 

Though the project no longer obliges Mark to start from scratches for a hard assignment as it has come to an end six years ago, he sets no limit on what his team can achieve by "hoping these guys to become scientists, not merely task-cracking engineers." Their research interests include the algorithm of ISOMAP to bridge discontinued motion pictures and Automatic Concert Video Mashup, a program to piece videos shot from different angles of a music concert together. These projects, as he points out with affective gratitude, are continuation of what was assigned to his team during the national program. 

The system framework for video mashup. (Image courtesy of Mark Liao)

In any case, Mark could not have been able to shun from the "mandatory research" that kept him "bloody tired" for his job's sake in the old days. So the only way out, he concludes, is to blaze one of his own, one that he construes as what he thinks of as the best part of the whole undertaking.

Not to mention the honorary titles, patents, technological transfer and entry in authoritative journals that came to be associated with Mark Liao, if we have a mundane look at what the nerve-wracking mission has amounted to – he has found his way out by nailing the job.