gmane.science.linguistics.wikipedia.wikitext

Collaborative Real-time Editing in VisualEditor

Abhishek Das — 2013-04-26T03:36:24

Hi!

I am Abhishek Das, a student developer from Indian Institute of Technology Roorkee, India. I have been understanding MediaWiki's code and have started contributing by solving a few bugs marked easy. Here is my user page: http://www.mediawiki.org/wiki/User:Abhshkdz

I have wide experience in building applications in PHP, MySQL & JavaScript (both client-side & server-side: Node.js). You can see my Github profile here: https://github.com/abhshkdz.

Recently, I have been working on a lot of real-time web applications using server set events, websockets as well as Socket.IO. I have experience in building collaborative document editors. This is a video conferencing with collaborative document editing application that I built during a 24-hour hackathon and won it: https://github.com/abhshkdz/hackview. It uses webRTC for Peer-to-peer video conferencing, and collaborative doc editing over shareJS.

Few days back, I built an open-source version of WorkFlowy (http://workflowy.com) using Backbone.js and Socket.IO: https://github.com/abhshkdz/HackFlowy. It has been gaining a lot of traction on Github with lots of stars and forks. I'm really excited about this one :D. The tasks get synced real-time using socket.io and mysql is used as the database.

I wanted to implement a real-time collaborative editor in Wikimedia's VisualEditor as well.

As pointed out by Sumana Harihareswara some work has already been done on this. I wanted to know what all is under development at the moment and how I can contribute in making it perfect.

Looking forward to hearing from you soon.

Thanks
Abhishek Das

Abhishek Das
B. Tech. (2nd year)
Electrical Engineering
IIT Roorkee
_______________________________________________
Wikitext-l mailing list
Wikitext-l< at >lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

IRC chats this week for Q&A about Lua scripting onwikis

Sumana Harihareswara — 2013-03-17T12:58:16

Hi, all!

As you might have seen on the Wikimedia tech blog
<https://blog.wikimedia.org/2013/03/11/lua-templates-faster-more-flexible-pages/>
or the tech ambassadors list
<http://lists.wikimedia.org/pipermail/wikitech-ambassadors/2013-March/000171.html>,
we enabled Lua on all Wikimedia sites last week. Lua
<https://www.mediawiki.org/wiki/Lua> is a scripting language that
enables wiki editors to write faster and more powerful MediaWiki
templates.  We're pretty excited about this, partly because of the
promise of Lua + Wikidata
<https://blog.wikimedia.org/2013/03/14/what-lua-scripting-means-wikimedia-open-source/>.
 To enable Lua templating on your own wiki, install the Scribunto
extension <https://www.mediawiki.org/wiki/Extension:Scribunto>.

If you have questions about how to convert existing templates to Lua (or
how to create new ones), we'll be holding two support sessions on IRC
this week: one on Wednesday
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=02&min=00&sec=0&day=20&month=03&year=2013>(for
Oceania, Asia & America) and one on Friday
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=18&min=00&sec=0&day=22&month=03&year=2013>
(for Europe, Africa & America); see m:IRC office hours
<https://meta.wikimedia.org/wiki/IRC_office_hours> for details. If you
can't make it, you can also get help at mw:Talk:Lua scripting
<https://www.mediawiki.org/wiki/Talk:Lua_scripting>.

If you'd like to learn about this kind of event earlier in advance,
consider becoming a Tech ambassador
<https://meta.wikimedia.org/wiki/Tech/Ambassadors> by subscribing to the
mailing
list <https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors>.

Thanks,
Sumana

Parsoid blog post & job opening

Gabriel Wicke — 2013-03-04T20:01:40

Hi,

we just published a blog post about Parsoid at

http://blog.wikimedia.org/2013/03/04/parsoid-how-wikipedia-catches-up-with-the-web/

We are also looking for somebody to join us in our Parsoid adventure:

http://hire.jobvite.com/Jobvite/Job.aspx?j=oIsbXfw2&c=qSa9VfwQ

Cheers,

Gabriel

_______________________________________________
Wikitext-l mailing list
Wikitext-l< at >lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

Getting started hacking parsoid

Andrew Dunbar — 2013-02-25T12:03:21

I'd like to get involved in the parsoid effort.

I've been hanging out  in the IRC channel on freenode but there's usually
just a dozen lurkers and no action.

In the mailing lists parsoid seems to be mentioned about as often in
wikitext-l and wikitech-l - which one is best to ask questions of this
nature?

I'd like to scratch my own itch rather than necessarily go after things on
the todo list and roadmap.

Basically I'm interested in what parsoid can do for parsing wikitext markup
into HTML (or other formats).

I want to use it without a mediawiki install and without an internet
connection. I see there is already some kind of support for reading in
articles from compressed dump files.

Any suggestions where I should start or where I can hang out to chat live
with people who could help getting me involved?

Andrew Dunbar (hippietrail)
_______________________________________________
Wikitext-l mailing list
Wikitext-l< at >lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

VisualEditor update

Sumana Harihareswara — 2013-02-16T18:48:23

Earlier this month, the Visual Editor team gave an update on their
progress and their goals for 2013.  Check it out:

https://meta.wikimedia.org/wiki/File:VisualEditor-Parsoid_-_2013-02_Metrics_deck.pdf

https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/2013-02-07
has the video on Commons & YouTube.  Significantly: the Visual Editor
*should* be able to work in Internet Explorer and to handle core basic
templates, such as {{cite web}}, by 1 July 2013.

Anyone want to give a talk to London Lua?

Sumana Harihareswara — 2012-11-19T18:02:28

http://www.londonlua.org/

I emailed a bit with Justin from London Lua and he expressed interest in
having a Wikimedian present, either in person or remotely, about what
Wikimedia's going to be doing with Lua.  There is already the video tutorial

https://commons.wikimedia.org/wiki/File:Berlin_Hackathon_2012_-_Lua_introduction_%28Tim_Starling%29.ogv

and the written tutorial

https://www.mediawiki.org/wiki/Lua_scripting/Tutorial

in case you want materials to use.  And if it would take a little money
for you to get to London, the Wikimedia Foundation might be able to
reimburse travel expenses:
https://meta.wikimedia.org/wiki/Participation:Support

Anyone want to give a talk to London Lua?

Sumana Harihareswara — 2012-11-19T18:02:28

http://www.londonlua.org/

I emailed a bit with Justin from London Lua and he expressed interest in
having a Wikimedian present, either in person or remotely, about what
Wikimedia's going to be doing with Lua.  There is already the video tutorial

https://commons.wikimedia.org/wiki/File:Berlin_Hackathon_2012_-_Lua_introduction_%28Tim_Starling%29.ogv

and the written tutorial

https://www.mediawiki.org/wiki/Lua_scripting/Tutorial

in case you want materials to use.  And if it would take a little money
for you to get to London, the Wikimedia Foundation might be able to
reimburse travel expenses:
https://meta.wikimedia.org/wiki/Participation:Support

VisualEditor Status for 3rd-party Wikis

Andrew Martin — 2012-08-24T13:29:29

Hello,

I was wondering if there was a way to enable to VisualEditor in the Main
namespace for 3rd-party wikis, or if an ETA is known for when users will
able to do that?

Thanks!
_______________________________________________
Wikitext-l mailing list
Wikitext-l< at >lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

Visual Editor / Parsoid demo release

Gabriel Wicke — 2012-06-26T13:02:39

Since this was not yet posted to wikitext-l:

Last Thursday, we rolled out a new release of the Visual Editor, the
first to use Parsoid to load and save actual wiki pages. You can read
about it at
http://blog.wikimedia.org/2012/06/21/help-us-shape-wikimedias-prototype-visual-editor/.

Or try it at
https://www.mediawiki.org/wiki/VisualEditor:Test

And please leave feedback at
https://www.mediawiki.org/wiki/Visual_editor/Feedback or in bugzilla.

Cheers,

Gabriel

Wikimedia at Open Source Bridge conference, late June,Portland

Sumana Harihareswara — 2012-06-19T10:36:58

June 26-29, a bunch of us will be in Portland, Oregon, USA for the Open
Source Bridge conference.

http://opensourcebridge.org/events/2012/schedule

WMF is sponsoring the Friday unconference day, and will host a hacking
table that day as well as (I hope) the Tuesday "Hacker Lounge
Project/Community Night."

Wikimedians are giving several talks during OSBridge:

    "Identity, Reputation and Gratitude: Designing for a community" by
Brandon Harris: Tuesday, 1:30
    "A snapshot of Open Source in West Africa" by Renaud Gaudin:
Tuesday, 3:45
    "Building A Visual Editor for Wikipedia" by Roan Kattouw and Trevor
Parscal: Tuesday, 4:45
    "Internationalization < at >Wikipedia: Helping add the next billion web
users" by Alolita Sharma: Wednesday, 10am
    "Why you need to host 100 new wikis just for yourself." by Ward
Cunningham: Wednesday, 2:30
    "Outreach Events: My Triumphs, My Mistakes" by Asheesh Laroia and
me: Thursday, 3:45

I give the opening keynote address on Tuesday morning. My tentative
title: "Be Bold."

If you're in or near Portland and want to come, let me know; I might be
able to hook you up with a free conference pass.

Upcoming hackathon for experts AND newbies: Washington, DC, USA July 10-11

Sumana Harihareswara — 2012-06-18T19:29:16

This is a reminder that you're invited to the pre-Wikimania hackathon,
10-11 July in Washington, DC, USA:

https://wikimania2012.wikimedia.org/wiki/Hackathon

In order to come, you have to register for the Wikimania conference:

https://wikimania2012.wikimedia.org/wiki/Registration

(Unfortunately, the period for requesting scholarships is now over.)

At the hackathon, we'll have trainings and projects for novices, and we
welcome creators of all Wikimedia technologies -- MediaWiki, gadgets,
bots, mobile apps, you name it -- to hack on stuff together and teach
each other.

Hope to see you!

Lua workshop this fall: Reston, Virginia, USA,Nov 29-30

Sumana Harihareswara — 2012-06-13T11:14:26

http://www.lua.org/wshop12.html

"the main goal of the workshop is to allow the Lua community to get
together and meet in person and talk about the Lua language, its uses,
and its implementation.... We shall have a plenary talk by Roberto
Ierusalimschy (Lua's chief architect) and several contributed talks....
If you'd like to speak at the workshop, please send a tentative title
and a short abstract to the organizers."

If you think it would be useful for Wikimedia to make an appearance, go
ahead and submit an abstract or suggest someone to go.

Parsoid web service

Gabriel Wicke — 2012-05-23T17:06:52

Hello,

I have just set up a very basic web service for Parsoid at

  http://parsoid.wmflabs.org/

It gives you the ability to browse the English Wikipedia as Parsoid
currently sees it, and provides forms for wikitext -> HTML DOM and HTML
DOM -> wikitext conversion tests.

Round-tripping is brand new and very limited, and does not support the
preservation of variable whitespace, templates and other complex
constructs yet. Currently 154 parser tests are passing in the new
--roundtrip mode that Subbu added last week.

Gabriel

Visual Editor & WikiEditor Integration outsideMediaWiki

Jim Safley — 2012-05-19T19:26:22

Will I be able to implement the Visual Editor outside MediaWiki? For
instance, could I install it as a JavaScript library in a separate
application?

I notice that the original edit toolbar and the toolbar that comes
with the WikiEditor extension are coupled so tightly to MediWiki that
it's near impossible to integrate them into a separate application.
(If I'm wrong, please educate me.) I wonder if the same will apply to
the rich-text editor.

Thanks,
Jim

Parsoid expanding [[:en:Barack Obama]]

Gabriel Wicke — 2012-05-03T15:04:30

Hello,

template expansion performance in Parsoid has made great strides
recently. [[:en:Barack Obama]] now expands in relatively reasonable time
and memory:

  echo '{{:Barack Obama}}' | time node parse.js  > obama.html

  42.18user 2.04system 0:46.23elapsed 95%CPU (0avgtext+0avgdata
  1528480maxresident)k
  848inputs+0outputs (8major+691968minor)pagefaults 0swaps

Sample output: http://dev.wikidev.net/gabriel/tmp/obama.html

The time utility reports the maximum resident memory inflated by a
factor of four (some bug in time afaik), so this works out to 382M
resident max which matches with top. This is measured on my Intel i3
M370 laptop, and in the same ballpark as the PHP parser running on the
cluster.

There are a few tokenizer issues visible in the output. Some of them,
such as the Navbox templates near the bottom, contain constructs like this:

{{#if:foo|<tr><th><th style="foo;|<th style="}} bar;">

Supporting this would be very messy, so the template should be changed
to use a saner nesting:

{{#if:foo|<tr><th>|}}<th style="{{#if:foo|foo;|}} bar">

We'll have to analyze how common this kind of mis-nesting is, and if we
can perhaps fix this up automatically. See also
http://www.mediawiki.org/wiki/Parsoid/Todo#Limitations.

Next bigger tasks I intend to tackle:

* Add full round-trip information, and preserve it through
transformations and DOM tree creation

* Call back to action=parse to retrieve information we need from the
wiki (link existence, image dimensions, many information-based parser
functions and magic words, extensions).

* Create a [ DOM -> token -> WikiText ] serializer chain based on the
existing serializer in the Visual Editor

Gabriel

Mime-Type for wikitext

Daniel Kinzler — 2012-04-25T16:09:31

Hi all!

I just noticed that MediaWiki uses two different mime types for wikitext:

* application/x-wiki is used by AjaxResponse, OutputPage and StreamFile.
* text/x-wiki is used by RawAction.php (i.e. when you use action=raw)

Is there a good reason for this, or is it just an oversight? I suggest to use
the same mime type everywhere, and keep the old one for compatibility reasons -
i.e. we should use application/x-wiki consistently, and RawAction could support
text/x-wiki as an alias.

That being said... the *correct* mime type would imho be
application/x-mediawiki. But i don't insist on it :)

Editing surface/UX: Inline menus

Erik Moeller — 2012-04-24T02:26:46

How can we make the Visual Editor as efficient as, or more so than,
wikitext editing when it comes to handling complex tasks like
citations, templates, etc.? I know it's a bit early to talk about
detailed aspects of the editing surface, but this may have some impact
on architectural considerations for plug-ins and such, so I wanted to
raise it on this list.

I don't think it's that much of a stretch goal to be more efficient
than wikitext editing for a lot of common cases. For example, using
complex templates manually is extremely inefficient, usually requiring
a visit to a Template: namespace to read the template documentation,
then apply the correct syntax and hope you're not making a typo in the
process.

A few models:

* Wikia's current-generation visual editor offers panels with access
to templates and categories, autocompleting as you type. If our goal
is to create an editing surface that focuses on the content and gets
out of the way (and IMO it should be), these types of panels add
clutter which is likely unacceptable.

* Most RTE editing surfaces have toolbars, and some also add desktop
application style pulldowns. The pulldown/toolbar combination has many
known usability issues as software complexity increases -- they become
cluttered, and keyboard shortcuts for anything but the stuff you use
all the time become hard to remember.

* Code IDEs use a lot of hinting/autocompletion based on intelligence
gathered from parsing the file you're editing, or any other files.
This works well when what gets rendered in the editing surface is
identical to what the user is typing. It's easy to auto-complete
"myObject.getSomeStuff()", but it's less discoverable that I should
start typing "{{Infobox country" to get a beautiful right-aligned
table.

* vi is the best example of a powerful modal editor which can perform
complex operations by essentially entering an in-editor command-line.
It's also well-known to be very difficult to learn and master, in
large part because its UX is so different from other applications
people commonly use.

What other models ought to be/are being considered?

A couple of ideas:

== Inline menus ==

You could have a menu key, say Ctrl+I or, if we want to be more
radical, a single compose key like "\", which triggers an in-editor
menu structure with associated keyboard hints.

E.g. if the user types <menu key>c, they see an overlay expanding at
cursor position:

    w - cite a website
    b - cite a book
    n - cite a newspaper
    j - cite a journal

So by typing: <menu key>cw, I could enter whichever the appropriate
dialog is for citing a website (which itself could be rendered in-line
if that gives us additional efficiency).

This type of system could intelligently trigger auto-completion where
appropriate. Say "y" is the shortcut for category, and I type:

    <menu key>yChurches in <cursor down><cursor down><cursor down><enter/tab>

to insert the category "Churches in the United Kingdom".

Or say "t" is short for template, so I could type:

    <menu key>tInfobox a<cursor down><cursor down><enter/tab>

to insert the template "Infobox album" and invoke the appropriate dialog.

One could make such a feature discoverable by adding hints to the UI
in appropriate places, e.g. if the most obvious invocation method is
through a tabbed dialog, each dialog could have a small hint
indicating how to quickly invoke it inline.

== Inline markup completion in visual mode ==

Another option would be to just intelligently detect use of existing
markup, e.g. to magically complete "[[Category:Churches in" or "{{Cite
web" or "{{Infobox co" even when in visual mode. This is beneficial
for people who already know wiki markup, but is arguably less
efficient/user-friendly than a menu/command structure that's designed
from scratch to be maximally efficient, discoverable and intuitive. It
also would tie us permanently to wiki markup.

What do you think? How can we make the visual editing surface
maximally efficient for frequent use?

All best,
Erik

Lua scripting update

Sumana Harihareswara — 2012-04-19T15:53:49

Just a quick note on how that work is going:
https://www.mediawiki.org/wiki/Lua_scripting#Status

WRT previous WYSIWYG editor: InlineEditor

Panos Louridas — 2012-04-18T07:01:51

Regarding the Inline Editor mentioned in today's post by Sumana:

(http://www.mediawiki.org/wiki/Extension:InlineEditor
http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/InlineEditor/)

To give credit where credit is due, the editor was the brainchild of Jan Paul Posma, then a student at Groniger, now a graduate student at Oxford, who coordinated our contribution. GRNET worked together with Jan Paul to carry out improvements as well as an interesting usability study that we presented in last year's Wikimedia Conference in Berlin.

We have since stopped working on the project since we were not sure that there was interest for it to be used. However, if there is a possibility that it may be useful in some way, we would be happy to see what could be done, discussing with Jan Paul.

Best Regards,

Panos Louridas
GRNET

_______________________________________________
Wikitext-l mailing list
Wikitext-l< at >lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitext-l

previous WYSIWYG editor: InlineEditor

Sumana Harihareswara — 2012-04-18T00:11:50

I was made aware of work done on an "inline editor" -- see
http://www.mediawiki.org/wiki/Extension:InlineEditor and code at
http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/InlineEditor/ .

This work was done by a team coordinated by GRNET (Greek Research
Network), which developed a WYSIWYG editor that produces MediaWiki syntax.

I presume that the Visual Editor team (Wikimedia Foundation and Wikia)
already know about this work as a predecessor to their own, but just
wanted to share it in case it's useful to anyone else while they wait
for the Visual Editor.

Userscripts for the VisualEditor?

Jonas Brekle — 2012-04-16T18:27:53

Hi,

I have not really looked into the code yet, but I just wanted to get
your oppinions on the question: Will there be a possiblity to integrate
userscripts or other extensions into the VisualEditor? Particulary i
would be interested in reusing the Parsoid and retrieve something like
"get parent heading from cursor"... I hope that makes sense :)

Regards 
Jonas