From cf51c3f0647d317a8b577dda011598a037ccac2f Mon Sep 17 00:00:00 2001 From: jave Date: Tue, 30 Dec 2014 16:20:29 +0100 Subject: [PATCH] replaced docs with external link --- README.xwidget | 1730 +----------------------------------------------- 1 file changed, 1 insertion(+), 1729 deletions(-) diff --git a/README.xwidget b/README.xwidget index ff060bf6b49..cfde411082b 100644 --- a/README.xwidget +++ b/README.xwidget @@ -1,1730 +1,2 @@ -*-org-*- -* Xwidgets - -This is an experimental branch to enable embedding of GTK widgets -inside an Emacs window. The Emacs abstraction is called an Xwidget, -for eXternal widget, and also in reference to the Xembed protocoll. - -There is a demo file called xwidget-test.el which shows some of the -possibilities. There are some screnshots at the emacswiki. - -Currently its possible to insert buttons, sliders, xembed widgets, and -webkit in the buffer. It works similar to the support for images in -Emacs. Adding more types of widgets should be fairly straightforward, -but will require adapter code for each type. - -A difference from images is that xwidgets live their own life. You -create them with an api, get a reference, and tie them to a particular -buffer with a display spec. - -Each xwidget can have several views. In MVC terms, an xwidget is the -model, and an xwidget-view is a view of the xwidget in a particular -Emacs window. - -The xwidget code attempts to keep the visual appearance of the views -in sync with through an Observer pattern implementation. This is -necessary to support the Emacs window paradigm. - -** building -bzr co bzr+ssh://bzr.savannah.gnu.org/emacs/xwidget/ -#or: -#git clone https://github.com/jave/xwidget-emacs.git -#the below compiler flags shouldn't be strictly necessary -export CFLAGS=" -g" -./configure --with-xwidgets --enable-asserts --with-x-toolkit=gtk3 -make -j4 -gdb -ex run src/emacs - -** try it out -If you have GTK3 and gtk-webkit installed, you should be able to -start the embedded webkit browser now: - -M-X xwidget-webkit-browse-url - -If that didnt work out try the minimal demonstration instead: - -(load-library "xwidget-test") -(xwidget-demo-a-button) - -It looks unimpressive, but it's a gtk button inside an Emacs buffer! -*** webkit hints -If you got webkit working, great! Please note, though, that the -current support is far from a full fledged browser. My focus is on -delivering a component that can be used to build a full emacs based -browser on. Since I implement a browse-url function you can get quite -far just by: - -(setq browse-url-browser-function 'xwidget-webkit-browse-url) - -then all Emacs browser interface systems work to a degree. -heres stuff I use currenly - -- m-x anything-surfraw interfaces to search engines -- C-o in org mode opens links inside org -- m-x ffap opens links anywhere in a buffer -- m-x gtk-lookup-symbol searches gtk docs -- m-x bookmark-jump -etc.. - -I'll add more examples as I go along. - -However theres lots of support missing, see TODO list for -information. Briefly: -- download handling -- keyboard field navigation -- isearch -- history -- sites that use flash. I dont really care about this issue so its -unlikely to be fixed. Just a heads up. - -** Stability -Beginning with Summer 2011 I am now able to use Xwidget Emacs as my -primary Emacs. That is good for the project and the stability of the -code. - -At the time of writing I have 24 hour Emacs uptime with several -embedded webkit browsers, Gnus, org-mode, tramp, etc. etc. - -That said, there are still many improvements that needs to be done, -particularily in memory management. Expect xwidget emacs to leak -heavily for now. - -** timeline for inclusion in trunk -The Emacs 24 feature freeze is passed, so xwidgets won't probably be merged -until Emacs 25. OTOH since I now use xwidget emacs as my primary -emacs, I will merge from trunk much more often than in the past. - -** reporting bugs -Emacs xwidgets uses the same tracker as mainline emacs, but a -different package. To report a bug: -M-x report-xwidget-bug - - -browse bugs: -http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs-xwidgets -** Thanks -emacs-devel@gnu.org. There are very helpful people there. When I -started the xwidget project I had no clue about the Emacs internals. - -* Brief overview of how xwidgets work -Xwidgets work in one way like images in Emacs. You bind a display spec very -similar to an image display spec to buffer contents. The display engine will -notice the display spec and try to display the xwidget there. The display engine -prepares space at the right place for the xwidget and so on for free, as long as -we provide proper sizes and so on back to the redisplay engine. - -** Issues -The problem is that Emacs cant actually draw the widgets, as it can with -images. Emacs must notify GTK about where the widgets should be, and how they -should be clipped and so on, and this information must be given to GTK -synchronous with Emacs display changes. Ok, so why is that difficult then? - -- How do we know when a widget is NOT to be drawn? The only way I found so far - is having a flag for each xwdiget, that is reset before a redisplay. When an - xwidget is encountered during display, the flag is set. After redisplay, - iterate all xwidgets and hide those which hasnt been displayed. - -- The gtk socket type for embedding external applications is desirable - but presents a lot of difficulties of its own. One difficulty is - deciding which input events to forward, and when and how to do it. - -** placement and clipping -the entire emacs frame is a gtk window. we use the fixed layout -manager to place xwidgets on the frame. coordinates are supplied by -the emacs display engine. widgets are placed inside an intermediate -window, called the widgetwindow. the widgetwindows are placed on the -emacs frame. - -this way was chosen to simplify clipping of the widgets against emacs -window borders. - - -** different strategies -Integrating toolkit widgets(gtk in this case) and the emacs display -engine is more difficult than your plain average gui application, and -different strategies has been tested and will continue to be tested. - -There was a distinction between live xwidgets and -phantom xwidgets, previous to the change to MVC. - -- the first aproach was to have the live xwidget on-screen, and move - them about. the phantoms were generated by snapshoting the live - xwidget. - -the drawback of that aproach was that the gtk toolkit is admirably -lazy and doesnt draw the widget if its not actualy shown, meaning that -the snapshots for the phantoms will show garbage. - -- the second aproach was to use composition support. that tells gtk - that the widget should be drawn in an off-screen buffer and drawn on - screen by the application. - -this has the primary advantage that the snapshot is always -available, and enables the possibility of more eye-candy like drawing -live and phantom widgets in different colors. - -the drawback is that its our own responsibility to handle drawing, -which puts more of the display optimization burden on us. - -this is aproach worked so-so. - -- another aproach is to have both live and phantom widgets drawn - on-screen by proxy gtk objects. the live xwidget will be entirely - handled in an off-screen window, and the proxy objects will redirect - events there. - -- combine on-screen and off-screen aproaches. maybe composition is the - way to go for most cases, but on-screen xembeding is the way to go - for particular special cases, like showing video in a - window. off-screen rendering and whatnot, is not efficient in that - particular case, and the user will simply have to accept that the - phantom of a video widget isnt particularily beautiful. - -- The current and seemingly sanest aproach implements a MVC pattern. - -** Testing -;;test like: -;; cd /path/to/xwidgets-emacs-dir -;; make all&& src/emacs -q --eval "(progn (load \"`pwd`/lisp/xwidget-test.el\") (xwidget-demo-basic))" - -** MVC and Xembedd -The MVC approach appears to be at least in principle robust for plain gtk -widgets. For the interesting case of gtk sockets which implements an -xembed host widget that allows for embedding other applications inside -an Emacs window, the story gets more complex. - -The problem is that xembed is designed to plug an application window -inside a socket and thats it. You can't move a plug between -sockets. I tried numerous hacks to get around this but there is -nothing that works really well. - -Therefore the Emacs part of the code will only expose well-defined -interfaces. cooperating applications will be able to use the interface -in a well defined manner. The problem is that there is no known xembeddable -application that implement the needed type of functionality, which is -allowing for creating new windows on the fly that plug into new -sockets. - -Therefore I will attempt to provide an external application that wraps -another application and through hacks attempts to provide the needed -multi view xembed function. That way Emacs is sane and the insanity -contained. - -This app will work by providing a socket that an app plugs into. The -socket window is copied efficiently by means of composition to a -number of other windows, that are then plugged into the different -Emacs sockets. -** old notes from x_draw_xwidget_glyph_string - - BUG it seems this method for some reason is called with bad s->x and s->y sometimes. - When this happens the xwidget doesnt move on screen as it should. - This might be because of x_scroll_run. Emacs decides to scroll the screen by blitting sometimes. - then emacs doesnt try to actualy call the paint routines, which means this here code will never - run so the xwidget wont know it has been moved. - - Solved temporarily by never optimizing in try_window_reusing_current_matrix(). - - BUG the phantoming code doesnt work very well when the live xwidget is off screen. - you will get weirdo display artefacts. Composition ought to solve this, since that means the live window is - always available in an off-screen buffer. My current attempt at composition doesnt work properly however. - - //allocation debugging. the correct values cant be expected to show upp immediately, but eventually they should get to be ok - // this is because we dont know when the container gets around to do layout - //GtkAllocation galloc; - //gtk_widget_get_allocation(GTK_WIDGET (xv->widgetwindow), &galloc); - //printf("allocation %d %d , %d %d\n", galloc.x,galloc.y,galloc.width,galloc.height); - - -*** old notes about the old live/phantom scheme - - //TODO: - // 1) always draw live xwidget in selected window - // (2) if there were no live instances of the xwidget in selected window, also draw it live) - // 3) if there was a live xwidget previously, now phantom it. - - else - { - //ok, we are painting the xwidgets in non-selected window, so draw a phantom - //printf("draw phantom xwidget at:%d %d\n",x,y); - //xwidget_composite_draw_phantom (xw, x, y, clipx, clipy); //TODO MVC there will be very few cases of phantoming - } - - - atm this works as follows: only check if xwidgets are displayed in the - "selected window". if not, hide them or phantom them. - - this means valid cases like xwidgets being displayed only once in - non-selected windows, does not work well. they should also be visible - in that case not phantomed. - -* ToDo:s -** TODO webkit crash -[2013-04-13 Sat] seems to crash a lot on http://www.dilbert.com -Not always, but enough to be annoying. - -** TODO optimize drawing off large offscreen widgets -Currently I just allocate as large an area as the offscreen webkit -widget desires. This works well most of the time. But a HTML page -might in principle be of infinite height so there are cases where this -doesn't work too well. - -Heres a proposed strategy: -- never grow the offscreen webkit over xwidget-webkit-max-height -- allow for webkit to handle its own scrolling internally as well -- be more clever about when you have more than one emacs window - showing the same webkit instance. -- allow to grow the offscreen instance in steps rather than just - allocate the entire height at once - -** DONE again a trace - CLOSED: [2011-10-28 Fri 13:48] -[2011-08-23 Tue] -the hunch is that since I still hand-wave the view storage the array -can get out of synchronous. so maybe switching to a lisp structure -will help as it did for the model. Anyway, doesnt happen at all often. -*** the trace -(gdb) bt -#0 0x0000000000685304 in xwidget_touch (xv=0x0) at xwidget.c:1225 -#1 0x00000000006853e7 in xwidget_end_redisplay (w=0x11b42ca0, matrix= - 0xff9bf40) at xwidget.c:1272 -#2 0x000000000041cc31 in update_window (w=0x11b42ca0, force_p=0) - at dispnew.c:3705 -#3 0x000000000041c0e5 in update_window_tree (w=0x11b42ca0, force_p=0) - at dispnew.c:3331 -#4 0x000000000041be8b in update_frame (f=0x1682a50, force_p=0, - inhibit_hairy_id_p=0) at dispnew.c:3258 -#5 0x000000000045066f in redisplay_internal () at xdisp.c:12931 -#6 0x000000000044e210 in redisplay () at xdisp.c:12110 -#7 0x0000000000567e65 in read_char (commandflag=1, nmaps=7, maps= - 0x7fffffffc040, prev_event=12708226, used_mouse_menu=0x7fffffffc254, - end_time=0x0) at keyboard.c:2447 -#8 0x000000000057613c in read_key_sequence (keybuf=0x7fffffffc4a0, bufsize= - 30, prompt=12708226, dont_downcase_last=0, can_return_switch_frame=1, - fix_current_buffer=1) at keyboard.c:9299 -#9 0x0000000000565d45 in command_loop_1 () at keyboard.c:1448 -#10 0x0000000000601008 in internal_condition_case (bfun= - 0x565962 , handlers=12760466, hfun=0x565259 ) - at eval.c:1490 -#11 0x0000000000565659 in command_loop_2 (ignore=12708226) at keyboard.c:1159 -#12 0x0000000000600992 in internal_catch (tag=12873826, func= ----Type to continue, or q to quit--- - 0x565633 , arg=12708226) at eval.c:1247 -#13 0x00000000005655bd in command_loop () at keyboard.c:1124 -#14 0x0000000000564da7 in recursive_edit_1 () at keyboard.c:759 -#15 0x0000000000564f43 in Frecursive_edit () at keyboard.c:823 -#16 0x000000000060444f in Ffuncall (nargs=1, args=0x7fffffffca20) - at eval.c:2986 -#17 0x00000000006507f8 in exec_byte_code (bytestr=145172929, vector=145179445, - maxdepth=116, args_template=12708226, nargs=0, args=0x0) at bytecode.c:785 -#18 0x0000000000604eec in funcall_lambda (fun=140575909, nargs=2, arg_vector= - 0x7fffffffcfe8) at eval.c:3220 -#19 0x000000000060467e in Ffuncall (nargs=3, args=0x7fffffffcfe0) - at eval.c:3038 -#20 0x00000000006035fc in Fapply (nargs=2, args=0x7fffffffd0b0) at eval.c:2494 -#21 0x0000000000603b43 in apply1 (fn=12874242, arg=301666310) at eval.c:2732 -#22 0x00000000005feb25 in call_debugger (arg=301666310) at eval.c:220 -#23 0x0000000000601ca9 in maybe_call_debugger (conditions=9431542, sig= - 12761282, data=301666742) at eval.c:1893 -#24 0x0000000000601785 in Fsignal (error_symbol=12761282, data=301666742) - at eval.c:1714 -#25 0x0000000000601898 in xsignal (error_symbol=12761282, data=301666742) - at eval.c:1749 -#26 0x0000000000601926 in xsignal2 (error_symbol=12761282, arg1=102756373, - arg2=0) at eval.c:1770 ----Type to continue, or q to quit--- -#27 0x0000000000604d6e in funcall_lambda (fun=102756373, nargs=0, arg_vector= - 0x7fffffffd398) at eval.c:3189 -#28 0x000000000060467e in Ffuncall (nargs=1, args=0x7fffffffd390) - at eval.c:3038 -#29 0x00000000006507f8 in exec_byte_code (bytestr=54783137, vector=109656229, - maxdepth=12, args_template=12708226, nargs=0, args=0x0) at bytecode.c:785 -#30 0x0000000000604eec in funcall_lambda (fun=109656517, nargs=0, arg_vector= - 0x7fffffffd890) at eval.c:3220 -#31 0x000000000060467e in Ffuncall (nargs=1, args=0x7fffffffd888) - at eval.c:3038 -#32 0x0000000000603b08 in apply1 (fn=109656517, arg=12708226) at eval.c:2725 -#33 0x00000000005fc8c9 in Fcall_interactively (function=109656517, record_flag= - 12708226, keys=12754549) at callint.c:379 -#34 0x00000000006044c2 in Ffuncall (nargs=4, args=0x7fffffffdc60) - at eval.c:2996 -#35 0x0000000000603c57 in call3 (fn=12893554, arg1=109656517, arg2=12708226, - arg3=12708226) at eval.c:2789 -#36 0x00000000005784cd in Fcommand_execute (cmd=109656517, record_flag= - 12708226, keys=12708226, special=12708226) at keyboard.c:10290 -#37 0x00000000005661fb in command_loop_1 () at keyboard.c:1575 -#38 0x0000000000601008 in internal_condition_case (bfun= - 0x565962 , handlers=12760466, hfun=0x565259 ) - at eval.c:1490 ----Type to continue, or q to quit--- -#39 0x0000000000565659 in command_loop_2 (ignore=12708226) at keyboard.c:1159 -#40 0x0000000000600992 in internal_catch (tag=12756258, func= - 0x565633 , arg=12708226) at eval.c:1247 -#41 0x000000000056560c in command_loop () at keyboard.c:1138 -#42 0x0000000000564da7 in recursive_edit_1 () at keyboard.c:759 -#43 0x0000000000564f43 in Frecursive_edit () at keyboard.c:823 -#44 0x0000000000563052 in main (argc=1, argv=0x7fffffffe678) at emacs.c:1711 - -Lisp Backtrace: -"recursive-edit" (0xffffca28) -"debug" (0xffffcfe8) -"image-bol" (0xffffd398) -0x68939c0 PVEC_COMPILED -"call-interactively" (0xffffdc68) -(gdb) - - -** DONE new annoying trace - CLOSED: [2011-08-13 Sat 16:16] -maybe related to scroll inhibiting or cursor inhibiting code. -It appears actually to be related to GLYPH_DEBUG=1. this flag is no -longer needed. -*** the trace -Breakpoint 1, abort () at emacs.c:383 -383 kill (getpid (), SIGABRT); -Missing separate debuginfos, use: debuginfo-install hunspell-1.2.15-2.fc15.x86_64 nss-mdns-0.10-9.fc15.x86_64 -(gdb) -(gdb) -(gdb) bt -#0 abort () at emacs.c:383 -#1 0x0000000000418f01 in matrix_row (matrix=0xac29400, row=-1) - at dispnew.c:1477 -#2 0x000000000046e113 in draw_glyphs (w=0x18235c0, x=198, row=0xa3af100, area= - TEXT_AREA, start=17, end=18, hl=DRAW_CURSOR, overlaps=0) at xdisp.c:22550 -#3 0x000000000047869f in draw_phys_cursor_glyph (w=0x18235c0, row=0xa3af100, - hl=DRAW_CURSOR) at xdisp.c:24882 -#4 0x00000000005083bb in x_draw_window_cursor (w=0x18235c0, glyph_row= - 0xa3af100, x=180, y=361, cursor_type=0, cursor_width=1, on_p=1, active_p=1) - at xterm.c:7440 -#5 0x00000000004790cd in display_and_set_cursor (w=0x18235c0, on=1, hpos=17, - vpos=19, x=180, y=361) at xdisp.c:25098 -#6 0x00000000004fa31f in x_update_window_end (w=0x18235c0, cursor_on_p=1, - mouse_face_overwritten_p=0) at xterm.c:644 -#7 0x000000000041ccb9 in update_window (w=0x18235c0, force_p=0) - at dispnew.c:3694 -#8 0x000000000041c165 in update_window_tree (w=0x18235c0, force_p=0) - at dispnew.c:3331 -#9 0x000000000041beee in update_frame (f=0x1658460, force_p=0, - inhibit_hairy_id_p=0) at dispnew.c:3258 -#10 0x0000000000450a2e in redisplay_internal () at xdisp.c:12983 -#11 0x000000000044e2a6 in redisplay () at xdisp.c:12099 -#12 0x000000000056a60d in read_char (commandflag=1, nmaps=6, maps= - -** DONE allow xwidgets to report their size - CLOSED: [2011-07-19 Tue 14:26] -now we just hard code sizes. but webkit widgets for instance can -report sizes that suit the content. support that. -** DONE BUG xwidget view ghosts - CLOSED: [2013-04-05 Fri 23:35] -(havent seen this in quite a while) -- xwidget-webkit-browse-url somewhere -- split window. -now theres 2 webkit views -- c-x 1 -now theres 2 views but one is a ghost! -one should have been deleted when its window died but that didnt work -for some reason here. - -- m-x xwidget-cleanup - -the ghost goes away because we killed explicitly but this is just a workaround. - -xwidget_view_delete_all_in_window(w); in delete-window-internal is not sufficient. -delete-other-windows-internal -delete_all_subwindows -unshow_buffer - -Added cleanup those window configuration hook which works in practice -but feels kludgy. - -*** code looks like this - -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -(defun xwidget-cleanup () - "Delete zombie xwidgets." - ;;its still pretty easy to trigger bugs with xwidgets. - ;;this function tries to implement a workaround - (interactive) - (xwidget-delete-zombies) ;;kill xviews who should have been deleted but stull linger - (redraw-display);;redraw display otherwise ghost of zombies will remain to haunt the screen - ) - - - -;;this is a workaround because I cant find the right place to put it in C -;;seems to work well in practice though -(add-hook 'window-configuration-change-hook 'xwidget-cleanup) - -*** but it ought rather to work like this -xwidget-delete-zombies should be called from C after window -configuration has changed but before redisplay. redisplay should not -be called. - - -** DONE BUG annoying backtrace - CLOSED: [2011-07-19 Tue 14:28] -(this no longer seems to happen even under heavy usage. seems merging -from trunk helped. lots were happening in redisplay at this time in trunk.) - -sadly happens a lot. -- happens even with no initialized xwidgets -- + row->glyphs[area][i].face_id -or similar code, so row is invalid for some reason. -xwidgets currently disable some redisplay opimizations so it might be -an actual emacs bug manifesting without optimizations. - -*** bt 1 - /* Compute the width of this line. */ - row->pixel_width = row->x; - for (i = 0; i < row->used[TEXT_AREA]; ++i) - row->pixel_width += row->glyphs[TEXT_AREA][i].pixel_width; - -(gdb) bt -#0 0x000000000045c340 in compute_line_metrics (it=0x7fffffff8a20) - at xdisp.c:17549 -#1 0x00000000004603da in display_line (it=0x7fffffff8a20) at xdisp.c:18792 -#2 0x0000000000457646 in try_window (window=23403045, pos=..., flags=1) - at xdisp.c:15399 -#3 0x00000000004559c9 in redisplay_window (window=23403045, just_this_one_p=0) - at xdisp.c:14944 -#4 0x0000000000450247 in redisplay_window_0 (window=23403045) at xdisp.c:13152 -#5 0x00000000005fdcd9 in internal_condition_case_1 (bfun= - 0x450208 , arg=23403045, handlers=12691046, hfun= - 0x4501d9 ) at eval.c:1538 -#6 0x00000000004501ba in redisplay_windows (window=23403045) at xdisp.c:13132 -#7 0x000000000044f19c in redisplay_internal () at xdisp.c:12706 -#8 0x000000000044f9f2 in redisplay_preserve_echo_area (from_where=7) - at xdisp.c:12964 -#9 0x0000000000568525 in swallow_events (do_display=1) at keyboard.c:4197 -#10 0x0000000000422554 in sit_for (timeout=40, reading=1, do_display=1) - at dispnew.c:5963 -#11 0x000000000056512c in read_char (commandflag=1, nmaps=8, maps= - 0x7fffffffd3f0, prev_event=12720514, used_mouse_menu=0x7fffffffd604, - end_time=0x0) at keyboard.c:2689 -#12 0x0000000000572c59 in read_key_sequence (keybuf=0x7fffffffd850, bufsize= - 30, prompt=12720514, dont_downcase_last=0, can_return_switch_frame=1, ----Type to continue, or q to quit--- - fix_current_buffer=1) at keyboard.c:9291 -#13 0x0000000000562897 in command_loop_1 () at keyboard.c:1446 -#14 0x00000000005fdb52 in internal_condition_case (bfun= - 0x5624b4 , handlers=12772898, hfun=0x561dab ) - at eval.c:1493 -#15 0x00000000005621ab in command_loop_2 (ignore=12720514) at keyboard.c:1157 -#16 0x00000000005fd4ce in internal_catch (tag=12768770, func= - 0x562185 , arg=12720514) at eval.c:1247 -#17 0x000000000056215e in command_loop () at keyboard.c:1136 -#18 0x00000000005618f9 in recursive_edit_1 () at keyboard.c:757 -#19 0x0000000000561a95 in Frecursive_edit () at keyboard.c:821 -#20 0x000000000055fba2 in main (argc=1, argv=0x7fffffffe188) at emacs.c:1704 - - -*** bt 2 - -** DONE Examine using XComposite rather than GTK off-screen - rendering. This would make xembed widgets work much better. This - would probably be rathter difficult, but could open up other - interesting possibilities for Emacs. There is an early attempt in - xwidget.c, but the X call to redirect to offscreen rendering fails - for unknown reasons. - - the attempt was further worked on, and the xlib calls replaced with - gdk calls, this works better. - - In the end I abandoned this aproach. Xwidget-osr is the new aproach. - -** TODO make the keyboard event code propagation code work. -There is an attempt to provide an api to send keyboard events to an -xwidget, but it doesnt currently work very well. - -*** TODO try gtk event creation instead -since that works fine in the webkit osr code. -but, oh no, that didn't work for some reason. -the widgets seems to receive the event but then the embedded widgets -hangs. - -http://kegel.com/gtk/button.c - -*** TODO examine some library to synthesize events -xdotool -xte xautomation -crikey -libxdo - -*** TODO webkit raw keyboard event escape -c-c tab could send a raw tab to the webkit instance. -** DONE remove the special-case for when the minibuffer is - active. I added some code to reduce the annoying problem display artefacts - when making the minibuffer the selected window. This made xwidgets in the - buffer go grey or black whenever one did m-x to activate the minibuffer. The - coded tried to handle the minibuffer as a special case. That simply wasnt a - good idea. Special-casing will never work properly. It is much better to spend - time finding solutions that work acceptably in the general case. - -** DONE disable emacs cursor drawing on top of an active xwidget. - This ought to be rather simple and should improve the visuals a lot. - -** TODO improve the xwidgets programming interface -so its less of hand-waving affair. This shouldnt be too hard, but I - have deliberatley not spent any time on it, since getting the - visuals right is much harder. Anyway, I sort of think the interface - should be somewhat like it is, except symbols is used instead of - integers. -*** DONE use symbols for xwidget types rather than ints - CLOSED: [2011-06-27 Mon 12:52] - - -*** TODO better lisp based structure for xwidgets -the lisp interface woud be like this: -- make-xwidget returns an xwidget object, similar to a process - object. this object is used when creating the display spec(instead of - the user defined id now used) - -the data structure would be something like this: -- a "process" like aproach to create the xwidgets. xwidgets are - coupled to buffers, somewhat like processes, except a buffer can - hold several xwidgets -- an xwidget has a plist to hold the model, like a process -- an xwidget has an assoc list of xwidget views - -there are some things that arent clear: -- an xwidget doesnt necessarily need to be coupled to a buffer but it - seems to be the clearest model. xwidgets would be buffer local -- xwidget-views are by necessity coupled to a emacs window so it might - be better to store them window locally rather than in an assoc - coupled to the xwidget model -- for some gtk widgets that resist an mvc approach, like the webkit - widgets, special operations are needed, similar to the old phantom - widgets aproach. so we need to differentiate live and phantom - instances for these troublesome widgets and let lisp manage all the trickery. - -stuff that needs to work: -- do something for all views of a xwidget(resize, value change) -- do something for all xw-views in an emacs window(deletion etc) -- lookup xw-view for xwidget in emacs window(during redisplay) -(- do something for all siblings of a xw-view. not atm) - -*** DONE xwidget creation interface - CLOSED: [2011-07-18 Mon 01:59] -xwidgets are a little bit like emacs processes but also a little bit -like emacs images. Therefore its not perfectly obvious how to handle -creation. Currently I just use hardcoded identifiers. the real scheme -needs to be something else. - -Heres a tentative approach: -- xwidget-create returns a xwidget object, like process creation - functions. the xwidget will be largely uninitialized until - discovered by redisplay. an xw belongs to a buffer -- xwidget-insert inserts the xwidget in a buffer. when discovered by - redisplay it will be initialized and a xwidget-view allocated -- an event will be emitted when initialization is finished when - relevant like for sockets - -the problem with this aproach is that its not really legal to reuse -xwidget objects by writing several display specs who reference the -same xwidget. It could presumably be done but it would just become -weird for no real benefit. the big preblem is that the display spec -decides the on-screen size, and its not sane to have xwidget views -with different sizes. therefore such display specs would need to be -catched and dissallowed. Except it is hard because AFAIK the specs -don't have an identity as such. A flag in the structure could be set -by lookup so the first display attempt would win. but then you can't -rewrite the spec to change the size. hmmm. A third approach would be -to just allow the 1st spec refering an xw during a redisplay to take -effect, the rest are signaled as errors. this wouldnt be too bad. - -the other aproach would be to work more like images: - -- write the display spec with all information needed to create the - xwidget. -- retrieve the xwidget objet from the spec with an xwidget-at-point function. It - can be uninitalized which client code must handle. Unlike - asynchronous process creation we dont get back a handle, because - there is none yet. -- emitted event on initialization, when needed. Many widgets don't - need this. for instance, a button sends an event when pressed. but - you can't press it unless its on screen, and then its initialized - properly. - -This approach seemed good, but how do I know which instance -generates an event if I cant set the id beforehand? - -so, therefore, the first two aproach is used. - - -*** DONE xwidget creation interface actually - CLOSED: [2011-07-18 Mon 01:59] -conclusion of above ramblings: -- should be similar to make-text-button -- don't init from display spec, instead during make-xwidget call -*** TODO callbacks would be nice -but they need to be handled initially with events for technical -reasons. C code can't call Lisp easily. The event handler can call the -callback though. - -** TODO more documentation -There should be user docs, and xwidget contributor docs. The current README -is all contributor docs there is now, apart from the code. - - - -** CANCELLED look into more ways of displaying xwidgets, like binding them to a - CLOSED: [2011-07-05 Tue 11:34] -window rather than a point in a buffer. This was suggested by Chidong. -This would be a useful addition to Emacs in itself, and would avoid nearly all -display issues. I still think the general case is more interesting, but this -special case should also be added. The xwidget would then be bound to -replace the view of a particular window, and it would only show in -that window. - -I got the webkit xwidget to work well enough so I dont see the need -for this now, except for sockets and I think it can better be dealt -with at the lisp level. - -** DONE MVC mode for xwidgets - CLOSED: [2011-06-27 Mon 12:53] -It appears unfruitful to chase using the same display mode for all -types of xwidgets. Composition is fun but not robust the way I -tried to do it. - -Instead there should be a set of MVC xwidgets. Each on-screen instance -of an MVC widget would be a real GTK widget. The instances would -communciate state using signals. - -There are drawbacks. There is no inbuilt support for MVC in GTK, so we -have to roll our own, which is tedious if not much work for the few -cases. - -MVC for xembedded application will need support from the applications -themselves. Inkscape supports multiple views to the same document, -other programs don't. In practice it might not be a big drawback. - - -*** DONE figure out what to do with the multiple frames case. - CLOSED: [2011-06-27 Mon 12:52] -This should be easier to solve with MVC. -Surprisingly, this just worked! -*** DONE how to propagate changes in views to other views? - CLOSED: [2011-06-27 Mon 12:53] -I used gtk signals, the implementation for sliders works well! - -** TODO canvas support -heres an interesting comparision of gtk canvases -http://live.gnome.org/ProjectRidley/CanvasOverview - -ATM there are small hardcoded demos in the code, these should be -removed and replaced with working xwgir counterparts. -*** goocanvas -goocanvas is a gtk canvas implemented using cairo. investigate. - -pros: -- it has a MVC model aproach out of the box which is nice. - -http://developer.gnome.org/goocanvas/unstable/goocanvas-model-view-canvas.html - -export CFLAGS="`pkg-config --cflags goocanvas` -DHAVE_GOOCANVAS" -export LDFLAGS=`pkg-config --libs goocanvas` -./configure -make - -I made a hello goo world xwidget so seems doable. -I wanted to load a SVG which wasnt immediately straightforward, so I -tried clutter. but it turns out the exact same strategy could be used -with goocanvas. - -*** clutter -maybe clutter can be used as a canvas? -pros: -- seems to have a lot of traction atm. many examples -- potentialy fast and cool vector graphics -cons: -- no out of the box MVC support, but seems doable. no worse than the - other home brew mvc support I have in xwidgets -(media-explorer in an application that employes the MVC pattern) - -http://www.openismus.com/documents/clutter_tutorial/0.9/docs/tutorial/html/sec-stage-widget.html - -there is also cool stuff like this: -http://gitorious.org/webkit-clutter/webkit-clutter which is an webkit actor for -clutter! hmmmmm. - -I want to render svg. aparently: - librsvg rsvg_handle_render_cairo(h, cr); - ClutterCairoTexture - Clutter - -export CFLAGS="`pkg-config --cflags clutter-gtk-1.0` -DHAVE_CLUTTER" -export LDFLAGS=`pkg-config --libs clutter-gtk-1.0` -./configure -make - -compiles but I get: -Gtk-ERROR **: GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in -the same process is not supported - -export CFLAGS="`pkg-config --cflags clutter-gtk-0.10` -DHAVE_CLUTTER" -export LDFLAGS=`pkg-config --libs clutter-gtk-0.10` -./configure -make - - -*** webkit html 5 -expose the DOM to lisp or something. The webkit xwidget works pretty -well now, so this might be the way ahead. -** DONE mvc code crashes after a while - CLOSED: [2011-07-12 Tue 18:52] -seemingly only when compiling with optimizations. -I have no idea why. - -Doesn't seem to happen after some code cleanups. -** DONE xwidget-resize-at - CLOSED: [2011-07-19 Tue 14:28] -reimplement so display spec is not involved -** DONE display spec validation - CLOSED: [2011-07-19 Tue 14:44] -it is an error to reuse xwidgets in several buffers or in the same -buffer. how do we catch these errors? -- showing the same xwidget twice in a buffer is no more wrong than - showing in several emacs windows, just conceptually wrong, so ignore - this case for now -- xwidgets now store a reference to the buffer they were created in, - so use that to invalidate xwidget references in oher buffers. but - thats not really an error either -- xwidgets should now be proper lisp objects so you dont delete them - you await their garbage collection. so therefore there can never be - invalid display specs - -so turned out this got solved by using proper lisp objects for -xwidgets. yay! - -** DONE clipping of controllers - CLOSED: [2011-07-05 Tue 11:33] - -Emacs uses a big GTK window and does its own clipping against Emacs -windows inside this area. So, in order to layout gtk widgets in emacs -windows we must clip thim ourselves. - -The following method worked well for a long time: -- make a gtk widget, say a button, xw -- make a clipping area, of type gtkfixed(many types have been tested) -- put the clip area in the main emacs gtk window -- figure out clip area changes during emacs redisplay - -the only weirdness was that one has to tell gtk the clip area has a -window in order to get clipping. This is weird because all gtkwidgets -are windows in a sense and a window is almost by definition also a -clipping area. - -Anyway, in GTK3 the gtk_widget_set_has_window(GTK_WIDGET ( -xv->widgetwindow), TRUE); call is ignored. - -The gtkeventbox which is documented to have its own window doesnt work -either. - -http://www.lanedo.com/~carlos/gtk3-doc/chap-drawing-model.html - -anyway clipping is rather complicated but seems to finally work okay. - -*** DONE subclass my own clipping widget - CLOSED: [2011-07-04 Mon 16:55] -http://www.lanedo.com/~carlos/gtk3-doc/GtkWidget.html#gtk-widget-set-has-window -mentions that it has_window can only be called inside a widget -implementation. - -this wasnt really the issue. allocation was the problem -*** DONE try scrolled window - CLOSED: [2011-07-01 Fri 10:56] -clipping does in fact work with -gtk_scrolled_window_add_with_viewport (xv->widgetwindow, xv->widget); -!! - -I get unwanted scrollbars in the widget though. - - gtk_scrolled_window_set_policy ( xv->widgetwindow, - GTK_POLICY_NEVER, GTK_POLICY_NEVER); - -stops clipping from working! - - -*** DONE try viewport - CLOSED: [2011-07-01 Fri 10:56] -gtkviewport is used in scrolled window so in order to remove -scrollbars it should be possible to use viewport directly. however, -viewport ignores size requests. or rather the container does. - - -*** DONE debug allocation - CLOSED: [2011-07-04 Mon 16:56] -the container determines how much size to allocate to child widgets. - - GtkAllocation galloc; - gtk_widget_get_allocation(GTK_WIDGET (xv->widgetwindow), &galloc); - printf("allocation %d %d , %d %d\n", galloc.x,galloc.y,galloc.width,galloc.height); - -after my clipping attemp shows that my size request is ignored! this -might be logical, since the container provided by emacs is a -gtkfixed. gtkfixed might choose to heed the widgets size desires and -allocate the entire widget size. but we want clipping! - -since i cant reasonably expect to change the emacs main container, i -can maybe overide the setallocation method in gwfixed, and adjust -allocation to clipping if its an xwidget asking for allocation. - -**** DONE subclass gtkfixed - CLOSED: [2011-07-04 Mon 16:56] -possibly i need to subclass gtkfixed and override -#+begin_src C - void gtk_widget_size_allocate (GtkWidget *widget, - GtkAllocation *allocation); -#+end_src -http://developer.gnome.org/gobject/stable/howto-gobject.html - -turns out emacs already does this for gtk3 according to jan D: ->>For GTK3, Emacs already subclasses GtkFixed, see emacsgtkfixed.[ch]. - -- widgets may not be underallocated, aparently -http://mail.gnome.org/archives/commits-list/2011-April/msg10950.html - -- how to call base class method/chain up -http://developer.gnome.org/gobject/stable/howto-gobject-chainup.html - -- the allocation modification could happen in the container or the - child. it feels more apropiate in the container - -it is however unexpectedy inconvenient to modify allocation because -the needed data is private to the base class. to overcome this: - - - run base class method 1st. - - then, iterate all children, and modify allocation for xwidget - children only. x y will then be set. - -JanD pointed out the GTK3 port already has its own subclass, so I -modified that one. - -*** DONE clip top - CLOSED: [2011-07-05 Tue 11:30] -there are four controller edges that potentialy need clipping. I begun -with right and bottom edges. clipping them is just a matter of setting -the right size of the widgetwindow and also ensure it gets the right -allocation from the container. - -clipping top (and left) is not equally straightforward. I'm using a -viewport now and scroll it the amount that needs to be clipped. -however, the viewport is sensitive to changes in allocation, which -makes it harder to use the allocation workarounds. - -see: -- gtk_widget_set_size_request -- gtkscrolledwindow - -I returned to using a simple gtkfixed for the widgetwindow. with -allocation hack and set_has_window it works. Idea prefer not to have -the allocatien hack and it wasnt needed it gtk3 only gtk2. needs -further investigation. - -** various code cleanups -There are many cleanups necessary before any hope of inclusion in -Emacs trunk. To begin with, the part of the patch that touches other -parts of emacs must be very clean. -*** DONE use FRAME_GTK_WIDGET (f) - CLOSED: [2011-07-20 Wed 20:02] -rather than gwfixed. - -*** DONE support configure - CLOSED: [2011-07-12 Tue 18:48] -*** DONE ifdef all xwidget code - CLOSED: [2011-08-13 Sat 16:19] -so you can reliably disable the code at compiletime -** DONE translate clicks - CLOSED: [2011-07-03 Sun 22:12] -on onscreen webkit peer to offscreen - -maybe -http://developer.gnome.org/gdk/stable/gdk-Windows.html#GdkWindow-from-embedder - -turned out to be not so hard, captured events, copied them and -forwarded them offscreen! - -** CANCELLED investigate gdk_window_redirect_to_drawable - CLOSED: [2013-04-05 Fri 23:37] -(cancelled this, the current approach seems okay) -http://developer.gnome.org/gdk/stable/gdk-Windows.html#gdk-offscreen-window-set-embedder -maybe could be used in place of my own copy hacks? to work it must -support a chain of redirects, which seems unlikely. the benefit would -be that I dont have to spend time optimizing redrawing. - - -** DONE remove xwidget_views when emacs window is deleted - CLOSED: [2011-07-05 Tue 11:29] -removing xwidget views when an Emacs window closes is not reliable. - -- switching buffers in a window seems to hide the corresponding - xwidget-views properly, but they might as well be deleted. - -- patching delete-window-internal could be used to delete the xwidget-views -this seems to work - - -** browser xwidget -although embedding a browser is not my primary concern many are -interested in this. some suitable browser component needs to be found -supporting gtk. - -*** DONE webkit - CLOSED: [2011-07-03 Sun 22:13] -there is a webkit gtk port. there is no obvious mvc support. -http://live.gnome.org/WebKitGtk -http://webkitgtk.org/ - -it might be possible to keep a set of webxits in artificial -synchronisation by recursive deep copy of the DOM from one webkit to -another. This will be error prone at best though. Another way might be -to just use bitmap copy of the "live"instance to the "phantom" -instances. the problem of transfering the live view remains though. - -export CFLAGS="`pkg-config --cflags webkit-1.0` -DHAVE_WEBKIT -g" -export LDFLAGS=`pkg-config --libs webkit-1.0` -./configure -make - -**** off screen rendering -export CFLAGS="`pkg-config --cflags webkit-1.0` -DHAVE_WEBKIT_OSR -g" -export LDFLAGS=`pkg-config --libs webkit-1.0` -./configure -make - -works a little bit but i get errors like: - -(emacs:8362): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11' - -set a breakpoint in g_log, backtrace seems to indicate -webkitViewportAttributesRecompute is the offender. - -maybe try gtk3 variants? -#+begin_src sh - export CFLAGS="`pkg-config --cflags webkitgtk-3.0 ` -DHAVE_WEBKIT_OSR " - export LDFLAGS=`pkg-config --libs webkitgtk-3.0 ` - ./configure --with-x-toolkit=gtk3 - make -#+end_src -crash in gtk_window_get_size instead. great. - -http://gtkplus-p3.0.sourcearchive.com/documentation/2.91.5-0ubuntu1/testoffscreenwindow_8c-source.html - -after many attempts, the basic issue remains. for some reason the -offscreen widget isnt ok when I want to snapshot it, so i simply get -emptiness. the surface is only ok sometimes. - -here is a useful debugging snippets: -#+begin_src C - // debugging redraw: - // - the bg colors always change, so theres no error in signal handling - // - i get this error now and then: - //(emacs:7109): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11' - // seems to happen in webkit actually. see README - - if(0){ //redraw debug hack. helped a lot in fact. use the with alpha painter below also - cairo_set_source_rgb(cr, osr_dbg_color, 1.0, 0.2); - cairo_rectangle(cr, 0,0, xw->width, xw->height); - cairo_fill(cr); - osr_dbg_color+=0.1; - if(osr_dbg_color>1.0) - osr_dbg_color=0.0; - - } -#+end_src -you need to terminate drawing like this: -#+begin_src C - //cairo_set_source_surface (cr, src_pixmap, 0,0); - //cairo_set_operator (cr, CAIRO_OPERATOR_OVER); - - //cairo_paint_with_alpha (cr, 1.0); - //cairo_paint(cr); -#+end_src - -the snippets change background color on oach redraw. - -**** on-screen rendering to separate window -an alternative might be to open a separate window and snapshot it. the -idea is that whatever oddness webkit does so that offscreen rendering -doesnt work, doesnt happen on-screen. the window could be opened -somewhere not in the way. - -*** CANCELLED firefox - CLOSED: [2011-07-03 Sun 22:13] -http://www-archive.mozilla.org/unix/gtk-embedding.html -seems to be severly bitrotted - -heres a newer aproach -http://hg.mozilla.org/incubator/embedding/file/29ac0fe51754/gtk/tests/test.cpp - -while webkit clearly has the best traction as an embedded, the -offscreen rendering issues makes it interesting to see what ff brings -to the table. - -turned out webkit has as good offscreen support as anyone, see I went -with that in the end. - - -*** DONE text field support - CLOSED: [2011-07-20 Wed 20:05] -Emacs captures all keyboard events so text field support isn't super -straightforward. - -**** propagate keyboard events -I have some old hacks for this and they are not good. -**** use the DOM model -expose document.activeElement to lisp. This is potentially more -interesting than just forwarding keyboard events. - -webkit_web_view_get_dom_document () - -this is hard it seems. an idea might be to hack elisp support for swig -to machine generate the bindings. -**** DONE inject javascript - CLOSED: [2011-07-03 Sun 22:50] -webkit_web_view_execute_script () - -this works now: -(xwidget-webkit-execute-script 5 "document.activeElement.value='test'") - -so it should be possible to do some interesting stuff. -execute-script does however not return anything at the interface level -so satisfaction is not total: - -http://markmail.org/message/4yowmdgras73z3x5 - -maybe -https://launchpad.net/gnome-seed - -or this funny hack: - im trying to understand how to interact via javascript to an embedded - webkit gtk instance [23:38] - i use webkit_web_view_execute_script() which is nice but doesnt return - a value, by design aparently [23:39] - any hints? - jave: afaik, webkit still doesn't have full gobject bindings [23:48] - jave: you can hack it up by making the JS modify the title, and read - the title from gtk-side - lucian: that was a pretty cool idea! - - -*** webkit_web_view_load_string () -I would like preview of html in a buffer rather than from uri. - - - -*** DONE simple xwidget-webkit wrapper - CLOSED: [2011-07-22 Fri 11:01] -so that it could be used for actual browsing :) -I dont want to reinvent too many wheels so i'd like to use existing -emacs facilities here possible. use bindings similar to w3m(or info) - -- m-x xwidget-webkit starts a session -- 'g' goes to a url -- use bookmark-jump i suppose. I mostly use org for bookmarks myself -- browse-url support so webkit can be the default browser -- some way of getting around the quirky keyboard interaction since - xwidgets dont receive keyboard events because I hawe no idea how to - do that in a sane way - -... and one can of course go on bikeshedding forever. lets keep it -simple and extensible, and compatible with other Emacs packages. - -the really cool ideas would need Emacs DOM integration, which is not -easy. - -** webkit related -*** TODO webkit support webkit signals - -**** DONE particularly document-load-finished - CLOSED: [2011-08-01 Mon 22:34] -http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-document-load-finished -because one might need tell set a title and sizes and things when it loads. -**** TODO event bug -Debugger entered--Lisp error: (error "Two bases given in one event") - -hapens sometimes with xwidget events. appears to be when the -originating xwidget is offscreen so that the event doesn't get caught -by the correct emacs event map. - -maybe I need to set the originating window in the event structure. - event.frame_or_window = Qnil; //frame; //how to get the frame here? //TODO i store it in the xwidget now - -since its an offscreen xwidget the buffer local keymap isnt the right -place for the handler. some global map should be used. - -onscreen widgets don't have the same issue. - -anyway, seems it'll turn out like this: -- xwidget-osr stores a callback and user data -- the event is an implementation detail only and get caught in the - topmost event map -- the event map calls the callback in the xw with the right args. - -we need the event handler at some level because we can't call lisp -asynchronously. - -**** TODO navigation signal -**** TODO new window signal -*** TODO console messages -http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-console-message -http://getfirebug.com/wiki/index.php/Console_API#console.count.28.5Btitle.5D.29 -because maybe we can make a simple JS REPL that way. - (xwidget-webkit-execute-script ( xwidget-webkit-last-session) - "console.log('hello')") -prints hello to stdout but theres no way to catch stdout from webkit I -think other than receiving the signal. - -*** TODO webkit flashkiller by default -while its possible to support plugins in the webkit xwidget, flash has -issues on 64 bit, and slows down emacs to a halt with off screen -rendering, and of course is not free software. its in the way for real -world usage even if its interesting to watch flash animations inside -emacs. which should be achieved with Gnash or other free software -instead. - -http://stackoverflow.com/questions/4885513/prevent-flash-in-cocoa-webview - -simply use this api: -http://webkitgtk.org/reference/WebKitWebPluginDatabase.html - -theres an implementation now but it's not robust enough webkit often -crashes taking emacs with it. - -*** TODO webkit downloads -when clicking a download link in Webkit Emacs should take over and handle it -from there. Probably need signals. There are Emacs libraries to -download things, with wget etc. an url.el facility should be made. -"download-requested" -*** TODO webkit alt-text not handled -XKCD use image-title to display a cartoon comment. These mysteriously -don't work ATM. Other mouseovers work though. Maybe webkit tries to -open a new window or something, which wont work. - -*** TODO webkit isearch in webkit buffers -have a look at how docview solves it -webkit_web_view_search_text () -*** TODO webkit relative references doesn't work -because we handle scrolling in a non-standard way. It does -work sort of when theres a html frameset and webkit scrolls by itself. - -internal links (page.html#section) do not work -see xwidget-webkit-show-named-element - -also did some webkit signal work for this. - -now it actually works! except for I need to know the Y coordinate of -the element to navigate to, and that can either be by "name" or "id" -attribute, currently "id" works. - - - -*** TODO webkit width adjustment handling issue -since there are so many levels of clipping and whatnot in xwidgets -sizing issues are difficult. - -- an xwidget is told how large it can be by emacs. thats the end of - it. if the xwidget thinks otherwise it will be clipped. -- but emacs can ask the xwidget how large it wants to be. it can then - resize the reserved area and inform the xwidget thusly. - -That should have been enough. but webkit never reports less than what -it already has. So currently a webkit view will only growth and not -adjust to smaller sizes. - -This is not a big problem in practice but is still annoying. - -to see the problem surface to http://www.slashdot.org -- xwidget-webkit-adjust-size -- xwidget-webkit-adjust-size-to-content - -and then compare by resizing in Epiphany, which is also webkit based. - -**** TODO try putting webkit osr inside a scrolling window -it seems webkit is supposed to behave differently while embedded in a -scrolling window. This is a bit cumbersome because the container stack -is already deep. -*** TODO xwidget webkit allow loading from string from emacs -*** DONE xwidget-webkit-last-session - CLOSED: [2011-08-01 Mon 22:38] -was rather hurried. end result is that the lisp layer only really -allows for one webkit session. -*** TODO extract DOM to lisp -then the SHR html renderer from Gnus could render the DOM as created -by Webkit. - -made a simple oxperimental DOM tree traverser. It can be expanded to -return a lisp representation, LDOM. - -in order to bring lisp and DOM closer together the LDOM can include a -mapping to the originating DOM node. so, find a node in LDOM, and the -cell maps to the original DOM. but since the LDOM is a copy it can get -out of sync. DOM events might help. -*** DONE C-X b in other buffer from webkit - CLOSED: [2011-08-12 Fri 22:20] -bafflingly resets the webkit view to the top. Maybe the window -reconfiguration hook code? further mystification is added because it -only seems to happen with ido mode enabled. - -in comparison with image-mode which does the right thing, I discovered -that image-mode has special code to handle scrolling. the browser mode -and image mode has some similarities. - -I made some delegation code frrom webkit mode to image mode. -*** TODO url-browse improvement -sindikat: site.com and http://site.com should be equivalent (simple site.com - throws error) - -Yes, but its unclear at what level in Emacs to do this -properly. I added a url-tidy function as a start. - -this should be further improved: -- change the call to url-tidy so its a hook -- provide a couple of demonstration hooks: - - url-tidy, which just prepends http:// - - youtube which appends &html5=1 to urls looking like http://www.youtube.com/watch?v=DZdUgjEx_dQ - - history which logs all visited urls like a traditional browser - -*** TODO sindicat notes -Here are some comments from user "sindikat" and my replies - -- http://ya.ru renders inadequatly (compare with any other browser) - - the search text-input is way below - -The problem is the size communication between Emacs and Webkit. - -- doing PageDown is endless; so if you do 100 PageDowns, you have to - do 100 PageUps to retun to the header of the page - -True, I hadn't noticed. Thanks. - -- http://linux.org.ru (just an example) renders incorrectly too - it - should stretch horizontally - -Size communication. - -- obviously, pointing of mouse over some link should change it to - pointing hand cursor - -Need to verify with some other webkit browser. - -- when you are somewhere on the middle of a long page, than go to some - other page, you are still in the middle, instead of being again on - the top - -This is because I inherit from Image view mode. I kind of like it so -we can add an option for it. - - -- changing dropdown menus cause flickering - - -- string entering is incorrect - by default it enters the title of the - page, while it should be empty - -The cause is the lack of return value in the webkit evaluation -API. Ive made some fixes. - -- internal links (page.html#section) do not work - - ive added a rudimentary function "xwidget-webkit-show-named-element" for this - -- maybe it's a good idea to implement Conkeror or some other - keybindings, where you press 'f' then select the exact where you want to enter text, without using mouse, - etc.; - -Indeed, this would require better DOM integration. - -- pressing 'home' and 'end' puts nonsense into minibuffer - -Probably because the Image mode derivative is mostly a hack. -fixed now I think. - - - - - - -- implement search (emacs internal isearch obviously doesn't work) - -Either use the webkit search but that doesn't feel right. It would be -better to expose the DOM and search that. - -- some sites intercept with keyboard; example - - http://www.artlebedev.ru/kovodstvo/business-lynch/2011/10/03/ uses - Ctrl+left/right/up/down to navigate between pages - this should be - implemented too - -Keyboard integration is the unloved step-child of xwidgets, unfortunately. - - - -** TODO xwidget image display spec compatibility -some history: the first version of the xwidget display spec was -the same as an image spec. This turned out not to be fantastic because -an xwidget is both like a process and like an image. it has a separate -existence from display. So now the xwidget display spec is just a -pointer to a xwidget. But then some useful functionality in Emacs -can't be reused for xwidget, in particular image-mode. - -Maybe a new image type could be added that was a wraper on an -xwidget. Then image mode could be reused for webkit mode. - -I tried some adaptor code in xwidget.el so webkit mode now delegates -to image mode but its a kludge. - -** socket related -*** TODO some flickering during redisplay of sockets -with gtk3 an size allocation workaround is used. -this seems maybe to result in flickering sizewize y-axis with the -xwidget socket type. The webkit xwidget doesn't seem similarily -afflicted. - -the size allocation workaround works by 1st running the ordinary -allocation then modifying the results. its done this way to minimize -the copy paste index from the base class. it might be that the -original allocation has a brief time window to show itself. - -tried to modify the allocation hack so it doesn't call allocate -twice. this doesn't seem to help flicker at all aparently so the -hypothesis falls. Maybe then a socket simply doesn't like being clipped -by gtkfixed. - -*** TODO xwidget view reaping too agressive -hide an emacs window for a while and return to it. the xwidget might -get reaped and a new socket thus created. -*** DONE try out OSR for sockets - CLOSED: [2011-07-25 Mon 21:30] - -didn't work too well in the inkscape case. it might be that some other -bitmap copy method works better though. - -basically sockets doesn't like to be offscreen because they want their -own gdk window. - -** DONE synchronise emacs background with xwidget color - CLOSED: [2011-08-11 Thu 11:04] -fine-tuning to reduce flicker. - -isn't needed if emacs bg erase disabled - -** DONE xwidgets doesn't work during bootstrap all of a sudden - CLOSED: [2011-08-01 Mon 22:33] -might be some annoying local issues with my install because it is not -reliably reproducible. (went away during merges) - -** CANCELLED low impact xwidget based image viewer - CLOSED: [2013-04-05 Fri 23:38] -(cancelled this because it no longer seems like a good idea) -for instance to render SVG using webkit, or some other canvas. -that way it would be possible to merge to trunk in stages. - -so, webkit could be used to display the SVG. the display spec for -images would be used. multiple webkits would be used rather than -offscreen rendering, so it would be GTK2 compatible. -** DONE xwidget movement doesn't work all of a sudden - CLOSED: [2011-08-11 Thu 11:03] -this used to work great. now it doesn't. - -suspects: -- XCopyArea - - x_shift_glyphs_for_insert - - x_scroll_run. this is run by the try_window* functions, and - inhibiting them doesnt help. but also callid in scrolling_window. - - -- try_window_reusing_current_matrix -- I used to enable GLYPH_DEBUG which I currently don't. it disables - many optimisations. this was fixed. -- lookup_xwidget then produce_xwidget_glyph gets called always but not -x_draw_xwidget_glyph_string probably because of scroll optimization. -movement detection could possibly be moved to produce_xwidget_glyph(not) - -no longer helps: -(setq inhibit-try-window-id t) -(setq inhibit-try-window-reusing t) - -workaround: -(run-with-timer 1 1 'redraw-display) - -seems to work: -inhibiting scrolling_window(). and this seem to be enough to restore the -old behaviour, GLYPH_DEBUG doesn't seem needed. - - -** DONE GLYPH_DEBUG doesn't work - CLOSED: [2011-08-08 Mon 17:30] -was stupid accidental line removal that was hard to spot -** TODO osc xwidget example -a couple of xwidget sliders that control a csound/supercollider song with osc. -so, for that to work we need slider callbacks to work. when a slider -changes send an osc message. use ocssend: - - oscsend localhost 7777 /sample/address iTfs 1 3.14 hello - -or better: -http://delysid.org/emacs/osc.el - -sliders could be defined in csound comments or something to illustrate -the point. or if real fanciness is desired parse the csound source -with Semantic and provide a control buffer corresponding to the -defined controls. - - - -Added: [2011-08-11 Thu 10:53] - - -** DONE SEB - CLOSED: [2011-10-26 Wed 15:36] -the SEB site does something funny so I can't insert text in -fields. aparently document.activeElement.value doesn't work with framesets. - -seems to work using the ugly javascript in -xwidget-webkit-activeelement-js - -** support downstreams -http://aur.archlinux.org/packages.php?ID=53902 -http://gpo.zugaina.org/app-editors/emacs-xwidget/ChangeLog -** DONE the proof of concept canvas code should be disabled by default. - CLOSED: [2011-10-12 Wed 23:03] -** TODO advi -active dvi viewer. investigate if it could be xwidgetified. -advi supports embedding inside presentations. -** cairo configuration support -gtk3 brings in cairo on Fedora, but apparently not on all plattforms. -pkg-config --cflags cairo -** TODO splint -splint -Demacs -DHAVE_CONFIG_H -I. -I/home/joakim/build_myprojs/emacsnew/emacs.bzr/xwidget.mint/src -I../lib -I/home/joakim/build_myprojs/emacsnew/emacs.bzr/xwidget.mint/src/../lib -DGSEAL_ENABLE -I/usr/include/gtk-3.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/freetype2 -I/usr/include/alsa -I/usr/include/librsvg-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/cairo -I/usr/include/libpng12 -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/ImageMagick -I/usr/include/libxml2 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -DGSEAL_ENABLE -I/usr/include/webkit-3.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/gtk-3.0 -I/usr/include/libsoup-2.4 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/libxml2 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -DORBIT2=1 -I/usr/include/gconf/2 -I/usr/include/orbit-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/freetype2 xwidget.c - -** TODO 32 bit bug -user reports that xwidgets segfaults on the 32 bit Mint distribution -but not the 64 bit. Mint is an Ubuntu derivative. I got some -VirtualBox images to test with. -** DONE youtube - CLOSED: [2011-11-01 Tue 11:19] -http://www.youtube.com/watch?v=DZdUgjEx_dQ&html5=1 -html5 makes it work without stupid flash plugins! -** TODO clicking on an webkit xwidgets -doesn't make the window active. this leads to problems. -** DONE "g" should default to current url - CLOSED: [2011-11-03 Thu 22:25] -"g" runs xwidget-webkit-browse-url which gets its interactive argument -from browse-url-interactive-arg. this might need a new optional argument. - -http://test -** TODO anything/helm support -hook so anything/helm can filter browser history. -** TODO new relative url code sometimes fail -http://www.dilbert.com -** TODO input field enhancements -*** password field. -was straightforward - -*** textarea -less straightforward. I would like it to work like emacs-w3m, where a -new editing buffer is opened. on c-c, the buffer is closed and the -browser field updated. however, it's not immediately obvious how to -store the reference to the field reliably. - -furthermore the current code doesn't seem to propagate linefeed -properly to text areas. - -** DONE bug in current navigation handler - CLOSED: [2011-11-09 Wed 10:04] -on www.dn.se -Debugger entered--Lisp error: (args-out-of-range "http://platform.twitter.com/widgets/hub.html" 54 357) - match-string(1 "http://platform.twitter.com/widgets/hub.html") - xwidget-webkit-callback(48890368 navigation-policy-decision-requested) - xwidget-event-handler() - call-interactively(xwidget-event-handler nil nil) -** TODO how to set the name of a webkit buffer? -not obvious because, the buffer isn't created immediately and there is -a callback that sets the buffer name automatically -** TODO how to find next field in tab order? -** TODO unique buffer names -the webkit xwidgets renames the buffer after load but not uniquely so -it sometimes fails. -** TODO kill the offscreen webkit xwidgets when last view killed -The offscreen xwidgets is currently kept around even if the xwidgets -views are all gone. this is a general problem and it requires actions -on the behalf of the application to resolve. - -In the case of webkit it is currently possible to get errors like these: - -Debugger entered--Lisp error: (error "Selecting deleted buffer") - xwidget-webkit-callback(60925380 navigation-policy-decision-requested) - xwidget-event-handler() - call-interactively(xwidget-event-handler nil nil) - - -because the last view is gone and the offscreen widgets is still -generating events. - -In the case of webkit it is okay to kill the offscreen widgets -completely when the user kills the last view window because it would -be unexpected by the user to see it pop up again. This is not true in -the general case. - -** DONE xwidgets debugging log - CLOSED: [2012-01-23 Mon 14:32] -currently theres a lot of debugging traces using "message" which is -annoying. Instead put them in a separate trace buffer. -(see xwidgetbuffer) -** TODO make garbage collect work for xwidgets -when an xwidget is removed from xwidget-alist, and there are no other -references(mostly views) the xwidget should be garbage collected. - -special finalization would go into gc_sweep() -** TODO embedding evince -http://developer.gnome.org/libevview/3.2/libevview-ev-view.html -would be useful for reading PDF:s and other document types. -it would work the same way webkit embedding works. - -** TODO support gobject introspection -https://live.gnome.org/GObjectIntrospection/ -supporting gobject introspection would mean that more gtk widgets -could be tried out with less effort, and also that the build process -and runtime support would be easier. The drawbacks are small: somewhat -slower execution, and difficulty in providing elisp bindings for -introspection. - -https://live.gnome.org/GObjectIntrospection/HowToWriteALanguageBinding - -http://developer.gnome.org/gi/unstable/gi-girepository.html -http://developer.gnome.org/gi/unstable/gi-overview.html - - -In order for GIR to work, it needs the namespace and class of a - widget. This is used to access the typelib file, which contains the - introspection data. The namespace and class is stored as a property - on the lisp symbol handle used by xwidgets to identify the widget - class. - -This snippet sets the needed :xwgir-class property, and calls the -set_zoom_level method: - -M-x xwidget-webkit-browse-url RET www.emacswiki.org RET - -Then eval the following: - -;;load the webkit typelib -(xwgir-require-namespace "WebKit" "2.0") - -;;provide the metadata needed so xwgir can work with the webkit-osr xwidget -(put 'webkit-osr :xwgir-class '("WebKit" "WebView")) -;;call the method -(xwgir-call-method (xwidget-at 1) "set_zoom_level" '(3.0)) - -It's also possible to create widgets dynamically, by using -introspection to call a widget constructor(from xwidget-test.el): - - -(defun xwgir-test () - (interactive) - (xwgir-require-namespace "Gtk" "3.0") - (put 'color-selection :xwgir-class '("Gtk" "ColorSelection")) - - (xwgir-demo-a-xwgir-button) - (xwgir-call-method (xwidget-at 1) "set_label" '( "xwgir set label!")) - ) - -Current limitation: -- Only 0 arg constructors are supported at the moment. Since xwidgets - defer construction, the args needs to be stored with the xwidget. - -- xwgir-call-method does indeed lisp to gobject conversion for the - arguments, but only some primitive types are supported atm. - -- next to no argument checking. If wrong type args are used with the - xwgir methods, emacs crashes. - -*** TODO xwgir create components with more advanced constructor -so this opens up an entire new can of beans. - -explain by example: -lets say we want to create agtkhscale on screen. its a slider. -https://developer.gnome.org/gtk3/stable/GtkHScale.html -we can already create buttons, so sliders shouldnt be much more -advanced right? wrong. - -the simplest slider constructor looks like: -GtkWidget * gtk_hscale_new -(GtkAdjustment *adjustment); - -so in order to call it, we must be able to forward arguments to the -constructor. this is almost already done, but we lack the ability to -pass object instances, only simple types atm. - -we need to be able to create GtkAdjustment -https://developer.gnome.org/gtk3/stable/GtkAdjustment.html - -this we can already almost do, because an xwidget is a gir object -with some decorations. we also store the decorated gir object in an -array, retrievable from lisp. - -In order for this to be usable in practice, we need some changes: -- lightweight objects should be stored un-decorated. they have no - need for the entire graphical machinery of xwidgets -- lightweight objects should be garbage collectable, but this is the - same for all the xwidget objects, and isnt really resolved atm. - - -** DONE investigate gdk_offscreen_window_set_embedder() - CLOSED: [2013-04-06 Sat 10:45] -https://developer.gnome.org/gdk/unstable/gdk-Windows.html - -,---- -| Offscreen windows are more general than composited windows, since they -| allow not only to modify the rendering of the child window onto its -| parent, but also to apply coordinate transformations. -| -| To integrate an offscreen window into a window hierarchy, one has to -| call gdk_offscreen_window_set_embedder() and handle a number of -| signals. The "pick-embedded-child" signal on the embedder window is -| used to select an offscreen child at given coordinates, and the -| "to-embedder" and "from-embedder" signals on the offscreen window are -| used to translate coordinates between the embedder and the offscreen -| window. -| -| For rendering an offscreen window onto its embedder, the contents of -| the offscreen window are available as a pixmap, via -| gdk_offscreen_window_get_pixmap(). -`---- - -okay, [2013-04-03 Wed] I finally suceeded in this approach! -it was pretty hard to make it work and currently works like this: -- the on screen dravwing area is the embedder -- you must implement "pick child" -event forwarding is done automatically! - -BUT its not really super, because it only works well with a single -embedder. - -perhaps the strategy could be refined: -- the window frame would be the embedder for all xwidgets. (but what - about several frames then?) -- in the from-embedder signal handler, which maps container coords to - embedded widget coords, find out which xw-view i clicked on, and - compute the coords. - -[2013-04-04 Thu] I had a strategy working for a xwgir button but not -a webkit. set_embedded in the motion event handler for the xv. it -even works for 2 frames! but not webkit :( - -[2013-04-05 Fri] it works for xwgir osr components, but not for -webkit. Webkit retains the previous event forwarding system. - -Now it works like this: -- the offscreen widget is created as before -- the on-screen views also as before, painting and copying as before. -- gdk_offscreen_window_set_embedder() is now used to embedd the - offscreen widget in the onscreen one, upon view creation -- only one widget can embedd one other. This means that the embedding - widget must be switched between the onscreen ones. This is now done - in the mouse motion event handler. - -The above approach has been tested for xwgir created buttons and seems -to work. it doesnt work for webkit, so the old scheme is preserved -for webkit. -** TODO investigate git-remote-bzr +Please see https://github.com/jave/xwidget-aux for documentation. -- 2.39.2