I moved away from using AR as a supplement for marketing or educational purposes, as I felt this was too close to my current job and day-to-day production, and I wanted to explore my skills and create something I was unfamiliar with. I wanted to create a graphic novel with a jovial story - something that was purely entertainment based, yet physically exhibited the use of combined technologies and printed media.
The value of the project is in the artefact itself, rather than it's agenda; I want the interaction of the user/audience with the story and the animation to be amusing, appealing, and give them a sense of awe or intrigue. My focus is on how narrative comes in many forms, and how contrasting technologies can contribute to storytelling. The printed story should be complete and cohesive without the AR interaction, however, it needs to add an element of surprise or appeal that engages an audience. In order to combine the virtual with the real world, I spitballed some storyline ideas and sketching in my sketchbook (right). This is written up below so it's legible: |
RAINFOREST
|
LONELY GHOST/ BAD GHOST
|
3-d scanning/printing workshop |
(AR) |
Current Practice - the graphic novel(LA) |
(PRINT) |
ROBERT HUNTER
Rob Hunter is a London based illustrator whose work and style inspired me and introduced me to the format of graphic novellas. The New Ghost - with its illustrative style, and whimsy, yet melancholic humanistic storyline - resonated with me and was the first of my collection of graphic novellas long before my personal practice developed. The ability to communicate effectively via illustration, and his use of varied panels, tonal colour, and traditional mediums/collage piqued my interest in alternative storytelling formats. Hunter's illustrations have a hand-made, dreamlike quality, utilising soft pencil lines, subdued blue hues, and rough, bleeding edges to panels. There is a sense of nostalgia, and connection to the maker, that makes his illustrations imbue a sense of emotion, or hidden feeling behind them. The New Ghost was my first introduction to Nobrow Press; an independent publishers that works with small artists and writers and focusses on producing high quality, artistic narratives. Their 17x28 Series (which The New Ghost is a part of) champions young graphic designers and emerging artists who wish to publish their stories - the only stipulation being that the story must conform to 24 pages, of 17x28cm. RYAN HESHKA
Heshka's Mean Girls Club struck me through its use of colour - utilising only greyscale and pink to reflect its subversive narrative of a gang of femme fatales committing crime, murder, drinking and riots. The illustration style subverts 1950's cliches using pin up figure references in vulgar and violent scenarios. His narrative and illustration style create a commentary on contemporary glamour; his audience can appreciate the feminist message beneath the superhero/nostalgic bravado of the action. Using familiar tropes/ artistic styles, portrayed in new and unsettling ways (through use of colour and characterisation) is an interesting way to push a narrative message. Playing with the audiences preconceived notions of similar artistic styles, forms of media, and storylines makes them question their own assumptions and attitudes. Heshka's work displays how visual language can be a powerful communicator - how the uncanny image can make an audience question their own perception. TOM HAUGOMAT
Through A Life by Tom Haugomat is a great example of how narrative and storytelling doesn't have to rely on words to be profound and emotive. This graphic novel is silent - but the structure, the limited colour palette, and the use of form; using reveals between overleaf pages, changing viewpoints between facing pages to add detail and pinpointing important events by expanding onto double page spreads - create a strong chronology, sense of time passing, and attachment to the story. The pages follow the general flow of the window, or object which the view is created through being on the left-hand page, followed by the first-person view created on the right-hand page. The hyper-focus on the key events in "Rodney’s" world puts greater emphasis on the audience to fill in the gaps between images, which ensures heightened interactivity with the page and a far stronger connection with the central character. Following the story of the character from birth through to death, the later pages mirror the earlier windows/views, in an oddly nostalgic, deja-vu-esque manor, which reinstills the idea of time passing, and the human notion of reminiscence of childhood and comfort in the familiar. The use of only 4 colours throughout helps relay the idea of the same lifetime being followed, despite changes in the characters image as he grows, and the disorienting viewpoints, and again highlights that Haugomat's choice of omitting details, and now colours, leans on the audience to draw their own conclusions. |
CHRIS WARE
Ware's gargantuan graphic novel Jimmy Corrigan: The Smartest Kid on Earth plays with form, function, and narrative to challenge storytelling tropes and question how "reading" is experienced. It was the first Graphic Novel to win a British literary award - making the Guardian's First Book Award in 2001. Published in 200 as a collected volume - after running in strips and serialised form for five years, the hardcover edition is an incredibly detailed maze of interjoining stories. It is interesting to note what has been included versus omitted in the formation of the printed book - and how this may impact the narrative. In the hardback edition, the inner covers of the book include a witty, tiny printed set of instructions as to how to navigate the novel, but these are so superfluous and in such a small font pt. size that they become irrelevant. Ware leaves the interpretation, and chronology of story/panels up to the audience - challenging them to draw their own conclusions. The dust jacket of the hardback edition also unfolds several times, creating its own, broadsheet comic book strip - emphasising physically that the compilation is made up of many interlocking, yet separate parts both in its form AND it's narrative. The orientation of the book changes throughout the novel, adding to the disorientation, and different panel sizes/styles frequently appear alongside each other. The detailed illustrative style and use of bold outline strokes is reminiscent of classic comic book style, but has a more modern, artistic feel. BRYAN LEE O'MALLEY
O'Malley is a notorious cartoonist, probably best know for the Scott Pilgrim series; a six volume series published between 2004 and 2010 in digest size black and white, before it's critical and commercial success instated a full-colour re-release, a successful cult movie, a video game adaption, and several soundtracks. The series was inspired by Plumbtree's 1998 single "Scott Pilgrim" - described by lead singer Carla Gillis as "positive but also bitter sweet". The lyrics " I've liked you for a thousand years" resonated with O'Malley and inspired him to create the story of 23 year old Scott - enraptured by delivery girl Ramona Flowers, and forced to defeat her seven evil exes if destined to be together. The relatable undertone of boy-meets-girl, and pursuit of love, keeps the story grounded and the audience captivated by waiting for the "happy ending", and helps add an element of realism underneath the extreme surrealist, "videogame" fight scenes, special powers, and pop-culture references. The story's origin in music, and O'Malley's own personal interest and background in music, is apparent in the characterisation of the band in the comic - and how sound and entertainment are portrayed in the silent print format. The rock and roll aspect of live music is showed through the sound being physically embodied, and drawn in great waves, lines, and zigzags - moving from the instruments to the audience/towards the page. The series was originally produced in black and white due to cost - it was much cheaper than working in full colour - however, O'Malley chose to use this to his advantage - "embracing the manga aesthetic" and leaning into the use of thick, black outlines, dramatic contrasts/reveals, and brush-stroke lines. ANDY POYIADGI
Lost Property by Andy Poyiadgi struck me through its use of colour palette, the hand illustrated pencil outlines of the characters which i thought mimicked my own intended style quite well, and the way its use of physciality/form mirrors the storyline. The graphic novella follows a postman - whose preoccupation with delivering other peoples' belongings has led to him losing most of his own possessions through life. These things, unbeknownst to him, have been collected, and stored in a lost and found. Confronted with all of the artefacts of his life, he chooses to make an art installation - sculptures created from a culmination of his life's objects. The idea of "objects" making up a character, or life, is an interesting commentary on how the physicality of things affects the human condition. The premise of losing things, or things becoming detached from us, is mirrored in the novellas physical format - the inner leaves are wire stitched together, however the outer covers are not attached to the rest of the novella. This embodies the idea that the two can be separated - one part "lost" from the other. This is an interesting concept, and goes to show how the form of the book can reinforce the written/illustrated story: when I first opened the book, I simply assumed this was an oversight/a mistake, but after reading the story, I realised it was a fun reflection of it. Poyiadgi also has a blog post about creating the illustration, character delveopment from initial sketches, and deciding on colours to be used, which I found useful for planning my own project. It is linked below: https://flyingeyebooks.com/creating-lost-property/ |
RISOGRAPHIC PRINTING - WORKSHOP |
(PRINT) |
CURRENT PRACTICE - AUGMENTED REALITY |
(AR) |
ARTHUR BRADLEY - Limina
Arthur was a previous MA Graphic Arts student, whose project utilised AR animation overlaying print. His focus was on educational, and advertising uses - creating Limina - an AR Magazine. I looked into his work on the suggestion of Stephen in one of our tutorials during this project, and it was useful to understand a) the scope of work created by another student for this and the following module, and b) the style of animation used. Most of his pieces rely on simply position, scale, and rotation of smaller overlay pieces, however, they are incredibly effective due to their interaction/relation to the printed page that they are attached to. Although the purpose or message of Arthur's project was not narrative based, one of his interactive elements was focussed around the idea of comic panels, and storytelling via animation through each panel - changing the perspective of the travel of a small boat, but keeping the subject the same so that continuity of journey was suggested. This was useful to look at, and gave me some ideas for how animation/interaction of AR elements between my own panels in the graphic novel might be used. It is also reassuring that another student on the course has questioned the same practices as I am aiming to with this project - how print and digital media can be used in tandem to create something with a more engaging feel, that will appeal to modern audiences. SUSI VETTA
Vetta is an AR artist who creates intricately layered animated artworks primarily in Adobe Aero. Taking inspiration from nature and architecture, Susi's augmented scenes interact with the existing space to create smaller fantasy worlds contained beyond the ability of the human perception. The mystical element to her work gives the augmented experience a magical, or dreamlike quality - the skewed, soft illustrated buildings, characters, and foliage in her scenes gently undulate. Most of her work is very soft, calming, and is built up of several layers placed along the "z-axis" infront of eachother. This layering of "flat" elements makes up the 3-D scene - and creates a sense of depth and dimension without the need for actual 3-D models. It is useful to understand how simulating the 3rd dimension can be achieved without 360 degree renders. Walking around the layers of the AR experience can place the user/viewer in the scene - although there is not a typical 3-D model, there is the ability to situate yourself between layers and interact with each aspect of the build. MATTHEW REY TREECE
Treece creates AR sculptures that explore alien architecture and post-modern form, whilst reflecting their natural surroundings colours and texture. The grounding of the sculptures in their settings adds realism to the augmented creations. Despite the viewers brain telling them that they are fantastical, impossible structures, the familiar materials and lighting projection convinces them of their physical stature and presence. In his own words, he "was drawn to augmented reality through a desire to see the manifestation of my imagination fused with our physical reality in a shareable format". Treece's work is an exemplar of how the impossible is made possible through use of Augmentation - his work takes the meaning of Augmented Reality literally - the sculptures ago one step beyond reality, using and improving on the materials and forms that we know and understand. They scratch the itch that audiences have of seeing constant new visual stimuli and extending the mind to problem solve how something is possible. Treece's work is created in Cinema4D and Adobe Dimension, and shared via Adobe Aero. |
STUART CAMPBELL
Campbell is an artist that uses AR and VR to curate new user experiences, landscapes, and immersive video and comics that challenge the boundaries between the real and the digital. Modern Polaxis - pictured above, is one of Stuart Campbells interactive comics that captured my attention when I was researching AR technologies available to me for my project, Little did i know that the Eyejacker app was created by Campell, and it is through this technology that Modern Polaxis is realised. In Campbell's own words, "a paranoid time traveller, comes to life as an Augmented Reality comic book. The book is presented as Polaxis private journal. Polaxis writes about his day to day life, but all his secret information, his paranoid delusions and conspiracy theories, he hides away in the Augmented Reality layer." Use of the augmented layer revealing a secret is appealing to a user - it gives an incentive to engage with the app and view the augmented reality - it gives a reason for the AR layer to exist beyond just novelty - it builds the narrative and reveals more of the context to the reader/viewer. The position of the character as a time-traveller also lends a hand to the suggestion of AR tech as a "futuristic", almost sci-fi development - arguably, it is considered a new, novelty invention, that has yet to reach its full potential. the suggestion of it being used in the commonplace - by means of a diary - suggests Campbell's, and indeed my own - hope that it has many great utilitarian and commonplace applications that we should be focussing on. NASA - First Woman
NASA released their first XR comic this year in September, as a recruitment incentive for their Artemis expedition, and to inspire more young women and people of colour to consider applications in the space programme, science and engineering. Using QR codes and a dedicated app, the reader can explore immersive VR and AR experiences, from minigames, to educational videos, to explorable lifesize environments. The use of new technology and video/games to appeal to a younger audience exemplifies the importance of engaging with a new wave of audiences - whose focus is more difficult to capture without the use of the digital aids. they are so accustomed to. The fact that NASA - a global pioneer of science, education, and human exploration are utilising the graphic novel and AR tech to promote their work reflects on it's importance, potential, and suggests it may be a prevalent form of storytelling in the future. Additionally, the choice to create a comic, focussing on a personal, human story to capture the hearts and minds of readers in an effort to garner recognition and applications to their workforce - highlights the power of emotive narrative, and the importance of a story being relatable, or some way connected to human experience. Whilst this project exemplifies what I myself am trying to achieve, and stands as a benchmark of interactive storytelling - it is important to recognise that the scale and budget of it is far beyond my own resources, and so creating something with the same production value, depth of interaction, or seamless utility is futile. It is important to keep my own goals and expectations in line with realistic targets. ANDREW WILSON
Wilson is a Bristol based lettering artist who experiments with creating 3-D animated text in procreate. Although most of his work concerns standard lettering, his understanding of depth, shadow, and imitating 3-D letters in signage and illustration lends itself to his work on actual 3-D text in AR. The concept that words aswell as images can be extended in AR expands the possibilities for AR uses in marketing, advertising, and communication. If language can be augmented and transformed in the digital space, it extends accessibility - letters can appear larger, or in a different language, or against a different coloured background - to aid reading. Additionally, making certain words or phrases stand out works in the same way as traditional painted signage, or electrified/neon light signs - it grabs the attention and creates a more engaging read. The use of procreate illustrates another "simpler" way to create 3-D assets than 3-d sculpting/modelling softwares - the ipad software is enabled with the ability to place objects along the z-axis - creating depth - easily, and basic animation options and creation of looping gifs are included. |
|
Adobe Aero is Adobe's answer to AR experiences, that can be accessed through the standard CC license and installed on ios or Android. The sharing is limited, and you cannot use QR to trigger experiences for other users - simply - you can simulate augmented elements in the space you are currently occupying/viewing with your camera. The app includes stock assets, and a variety of trigger options/animation that can be applied easily.
This first test was to see how I could insert my own vector illustrations into augmented reality. I created a simple 2-D pumpkin illustration in Adobe Illustrator, and saved as a .png. I then imported this into Adobe Aero, and applied animations based on triggers - making the pumpkin "bounce" and "spin" when tapped. The face of the png isn't tracked, so you can move around the object (you can set it to always face the camera as it moves) - however, it becomes apparent that it is a flat image, which I believe deters from the immersion of the experience - it simply looks as if it is placed over the video, rather than interacting with its environment. |
Pros:
|
When researching open-source software/apps for AR experiences, I found that Zapworks most accurately covered my initial idea to have QR/triggers on pages that a reader could scan in order to launch the animated overlays of the story. They offer several different options for creating AR experience - including their own £D animation studio, and 3D sculpting tech for more advanced pieces, however, there is a steep learning curve to using and creating experiences with this.
For this test, I was primarily concerned with testing the printed QR code as a trigger. The Qr code launches the web app via a link, which opens the users camera, and displays the augmented reality elements. I used the built in studio to add simple text, image, and video overlays. I was not concerned with the content, but the functionality. Cons:
|
|
This was a simple test that combined aspects of the previous two. I wanted to test out Aero's anchor to page/image function, and explore how 3-D assets appeared through the app - as I was unsatisfied with the view of the 2-D illustration from the previous test.
I used a page of my sketchbook as an impromptu trigger and anchor - the illustrated elements made is easily recognisable to the camera. I then positioned one of Adobes stock 3-D models (an armadillo) onto the page, and explored how this interacted with the scene when the camera and the page itself were moved. There appears to be some glitches in the positioning and angle of the 3-D model when anchored to the page, it is unsure of the angle of the flat surface, and when panning the camera, it struggles to keep up with the viewfinder. It fails to hold its position well when the page itself is rotated, and even when the camera is moved, it jumps between perspectives. This makes the model seem unrealistic, and does not make for an immersive experience. Whether this is due to the anchoring, or the 3-D model itself requires further testing. |
animation causing problems...Although drawing the panels traditionally, and reusing the outlines in photoshop seemed like it would cut corners, I ran into issues later when trying to animate. Namely - when objects moved, there was blank space, or gaps "behind" where they originally were , or lines were left unfinished (see video: right - corner of mirror, and space behind ghost). In addition the colour and definition of the outlines became increasingly spotty, and was not as polished at I would like it to be, or for it to work effectively in the digital space. I soon realised that I would need to create all of my illustrations in entirety digitally. The colouring process would remain the same, however, all outlining would need to be repeated (i.e: traced from the original pen drawing). Additionally, my workflow is further increased by the fact that my digital illustration is naturally slower than putting pen to paper - due to its unfamiliarity, and the superfluous options when it comes to materials, effects, and fills.
|
|
the book: thinking about form, function, and physicality |
(Print) |
There are countless forms of narrative in the world. . . . Moreover, in this infinite variety of forms, it is present at all times, in all places, in all societies. - Roland Barthes.
What is a book? A book is an experience… A book starts with an idea. And ends with a reader. - Julie Chen & Clifton Meador, How Books Work.
The page is a powerful interface between designer and reader, flexible enough to respond to a variety of demands while remaining comprehensible and communicative. - Bonnie Mak, How The Page Matters.
While contemporary books seem to come in all sorts of proportions, out associations with size reflect its role in the codex's early days: a large book, whether in dimensions or heft, suggest value - Amaranth Borsuk, The Book.
EXPERIMENTING WITH 3-D MODELLING |
(AR) |
In exploring the multitude of tabs and options on blender; attempting to find the texture properties, I found the sculpting window/console. To learn the tools and to complete my model, I went through each individual sculpting tool, and applied them to my smooth model, noting what effect each took for later reference. I managed to warp the shape of the cloth to create folds/ripples that mimicked my original illustrations, and added more volume at the back of the ghost so that he was no longer perfectly symmetrical. Once I was happy with all my manipulations, I exported my final model, below:
|
At this point, I had a smooth, simplified but recognisable ghost model that was hollow (left). As I had been following several tutorials to create a model, unknowingly, the "cloth" properties I had applied to the model actually only applied to animation in blender. Rather than simply being a texture render, (as I had expected), this meant that when my ghost interacted with another solid, it would crumple - as a cloth sheet would do, and disappear from the screen. I tested this (below) - adding a new solid - and creating an animation path to drop the ghost onto a platform. You can see that on impact, the cloth at the top of the ghost starts to rumple. I have tried adjusting setting so that a) the ghost doesn't disappear and remains in a heap on the platform and b) the cloth creases more dramatically, but I cannot troubleshoot it and get it to work. Since I have no intention of animating in blender, I chose to leave exploring these options, as they were not a productive use of my time for this particular project.
|
|
Results of first model and AR interaction testing:
|
Blender: Work in progress model - creating the "legs"
|
Testing AR - putting the model into Aero
|
|
|
I did attempt various workarounds - exporting with different settings (frame rate/ colour profile/ alpha etc.), as different file formats, a total of 7 different times. I tested the video by re-importing it back into After-Effects - where it would clearly display with the alpha channel - but for some reason would not play outside of Ae with this. I scoured the internet for workarounds - but all pointed me back to simply rendering as a .mov with RGB + Alpha enabled. I exhausted all of the options/ file formats available to me on both my laptop, and the Macs on campus.
|
|
All of these file formats continued to show the same issue - whether not playing on my quicktime player, or simply appearing with the same transparency issues/black background. At one point, I did question if I was simply mistaking the black background default of the .mov players as a solid background, but the problem persisted as the black background was being rendered in the Eyejack app - which was where the transparency was key. After spending several hours over a couple of days exporting various formats, I had around 27 different file formats exported of the same video - none of which were suitable to overlay in the app.
|
My next area of investigation was creating a separate animation test - as I was concerned that I had some settings I was unaware of included in my panel animation that were preventing the alpha channel from exporting correctly. I used a simple cube solid and animated its position and rotation. I exported in all of the above mentioned formats - through both the render queue and media encoder - however, the black background was still appearing in Eyejack when I tried to view the animated overlay. (right)
By this point, I had worked out the precise format (Quicktime - Apple ProRes 4444 with alpha) to export the .mov out using the Media Encoder to ensure transparency, and playability on my Macbook. This is below in an image for future reference: |
|
Page 1 Panels - Digital Test
|
Page 1 Panels - Print Sample
|
|
|