[Impromptu] Code Over Graphics (like the videos)

[Andrew Sorensen]

Hi Guys,

I've tried to address a couple of different answers with one code  
example, so hopefully the example below is not to confusing.  It does  
more than either of the previous emails required because it includes  
answers for both.  You don't need to use both opengl and a graphics  
canvas together, I've just done it here to demonstrate a few different  
things.  Have a play around with the code and you should get the idea.

In short there are many ways to composite stuff.  Use opengl, use  
coreimage, use bitmaps or use a combination of the three!  The easiest  
(but least efficient) way is to use bitmap images only.  The trick to  
bitmap images is to use the functions gfx:image2image, gfx:path2image,  
gfx:text2image.  They basically work the same way as their canvas  
equivalents gfx:draw-image, gfx:draw-path and gfx:draw-text.  You can  
then draw one image onto another with a transparency.  Basically you  
just keep drawing bitmaps on top of one another.

You can get images from lots of different places -

gfx:make-image (completely transparent image)
gfx:load-image (from a file)
gfx:get-code-image (from the editor window)
gfx:get-live-frame (from a camera)
gfx:get-image-from-canvas (from a graphics canvas)
gfx:get-image-from-nsview (from an nsview)
gfx:get-image-from-window (from a window)
gl:get-image-from-opengl (from an opengl context)

If you want to use gl:get-image-from-opengl your opengl code needs to  
include a call to gl:update-backing-image.  You need to include this  
in your opengl code so that a bitmap image of the current opengl  
context is captured.  You can then retrieve it using gl:get-image-from- 
opengl.  Note that you can also draw bitmaps to an opengl context - so  
you could draw (gfx:get-code-image) to an opengl context rather than  
drawing the opengl context to a canvas.

Now I have to emphasize that working with gfx:image2image is fairly  
inefficient.  It's easy to type but slow to execute.  Doing all the  
work in OpenGL or using CoreImage exclusively is much more efficient  
but requires more work/understanding.   Impromptu has quite a lot of  
support for doing OpenGL (GLSL etc.) and CoreImage (CIFilters etc.)  
programming.   However, if all you need is two layers (one for code  
and one for graphics), then just calling gfx:image2image is a nice  
simple solution.


(gfx:start-live-video)

(define canvas (gfx:make-canvas 640 480))

(define gl (gl:make-opengl))
(gl:open-opengl gl '(0 0 640 480))


;; draw code behind a rotating cube
;; if you wanted you could now call gfx:start-movie-capture
;; and pass the gl context where canvas usually goes.
(define loop-gl
    (lambda (beat)
       (let* ((code (gfx:get-code-image))
              (flipped (gfx:flip-image code))
              (width (car (gfx:get-image-size code)))
              (height (cdr (gfx:get-image-size code))))
          (gl:clear gl (+ *gl:color-buffer-bit* *gl:depth-buffer-bit*))
          (gl:window-pos gl 0 0 .99)
          (gl:draw-pixels gl width height *gl:rgba* *gl:unsigned-byte*  
flipped)
          (gl:load-identity gl)
          (gl:translate gl 0 0 .5)
          (gl:rotate gl (* beat 10) 1 1 1)
          (gl:color gl 1 1 1 1)
          (glut:wire-cube gl 0.5)
          (gl:flush gl)
          (gl:update-backing-image gl)
          (objc:release (+ (now) 5000) code flipped)
          (callback (*metro* (+ beat (* .5 1/6))) 'loop-gl (+ beat  
1/6)))))

(loop-gl (*metro* 'get-beat 4))8


;; define an image to use as layer3
(define layer3 (gfx:make-image 640 480))


;; draw paths to layer3
(define draw-on-layer3
    (lambda (beat)
       (gfx:clear-image layer3)
       (dotimes (i 10)
          (gfx:path2image (gfx:make-circle (cosr 320 200 (/ i 100))
                                           (sinr 240 200 (/ i 100))
                                           20)
                          layer3
                          '() (list (/ i 5) 0 1 1)))
       (callback (*metro* (+ beat (* .5 1/6))) 'draw-on-layer3 (+ beat  
1/6))))

(draw-on-layer3 (*metro* 'get-beat 4))


;; draw video and layer3 transparently into the opengl bitmap
;; then render the opengl bitmap to the canvas
;;
;; You could easily replace
;; (gl:get-image-from-opengl gl) with (gfx:get-code-image)
;; if you didn't want to use any opengl
(define capture-code
    (lambda (time)
       (let ((video (gfx:get-live-frame))
             (opengl (gl:get-image-from-opengl gl)))
          (gfx:image2image video opengl .3)
          (gfx:image2image layer3 opengl 1)
          (gfx:draw-image time canvas opengl .9)
          (objc:release (+ time 5000) opengl video))
       (callback (+ time 2000) 'capture-code (+ time 5000))))

(capture-code (now))

;; start-movie-capture takes a canvas OR an opengl context
;; so you could call (gfx:start-movie-capture gl ...
;; as long as your gl code calls gl:update-backing-image
(gfx:start-movie-capture canvas "~/tmp/my.mov" #t)
;(gfx:stop-movie-capture canvas)


Cheers,
Andrew.