Drag-and-drop objects into Storyline

Hei,

1. This article is about how to drag-and-drop objects from outside the browser onto runtime Stories.

1.1. This differs from moving objects on a Storyline slide onto another object. The drag and drop described here refers to dropping objects from outside the browser, such as from Windows Explorer, as input to a running story.

2. The approach I take to do this requires using Javascript. 

3. In my application I want to drop audio files from my laptop onto a Story input text field. You can use the same method to drop any kind of file, with modest extensions to the Javascript discussed here. 

4. Adding drag and drop turns out to be quite simple to do, although fairly coarse grained. It is possible to drop objects onto a Story window or document (DOM).

4.1 However, I was not successful in dropping objects onto individual Story elements (by dynamically attaching say 'div' objects to those runtime elements). This fine-grained approach would be nice, but it both breaks the Storyline runtime player and falls outside the scope of Articulate support. It would also be too complex for inexperienced programmer Story authors to follow. 

5. The result of the coarse grain approach is that when you drop an object onto a runtime browser Story, the Javascript that executes needs to work out which Story element wants the input.

5.1 In my application I have just one text input field on any slide / layer. So all my Javascript needs to do is access a Storyline variable, using GetPlayer().GetVar("..."), to determine where the input should go. In the attached story below, this is the variable "Running" set when the timeline starts on the Run layer. 

6. If you want to follow this description, see the attached story below and look at the Execute Javascript when the timeline starts trigger on the Run layer. 

7. There are three functions. Well four if you count processFile separately from dropHandler. We'll go over each of them. 

7.1. jukeboxDragAndDrop( ) which is executed by the very last line of this trigger code. It is run once, when the timeline starts on the Run layer by the Storyline trigger. 

7.1.1 jukeboxDragAndDrop attaches two "event listener" functions / handlers onto the document. A document is the top-most object in html / browser speak (DOM model), within which your Story runs.

7.1.2 The function dropHandler will be run when your runtime browser detects a drop event (this is when an object is dropped on the window, when you release the mouse button). 

7.1.3 The function dragOverHandler will be run when your runtime browser detects a dragover event (this is when you drag an object across the browser page with your mouse).

7.14 As an aside, console.log("..."); is a very useful debug tool for any Storyline author. The value of the string is displayed at runtime in the browser debug window. You can add console logs anywhere on your slides using execute Javascript triggers. I use them to debug plain old Story behaviour which I couldn't work out, to grasp what goes on when my slide "doesn't work" as I expect it to. Debugging the mis-logic in my head versus what the runtime does. 

7.2 dragOverHandler is the function attached to a "dragover" event. It is run whenever you drag an object with the mouse across the browser. 

7.2.1 All it does is prevent the default browser behaviour, by consuming the event "ev". In essence it does nothing. Which is what we want the browser to do. 

7.3 dropHandler is the function attached to a "drop" event. It is run whenever you release the mouse button to drop an object.

7.3.1 Here too, after a console log trace, the first thing it does is prevent the default browser behaviour. The default would generally be to open the file. And we don't want that.

7.3.2 We get the value of the Storyline variable, Running. This logic is used to determine which layer is running and hence which input field will receive the dropped value. Since I only have one input field per slide / layer. If you have multiple input fields on any slide / layer then you will need to modify this logic.

7.3.3 When it is run, dropHandler receives an object, ev, from the browser. It is the parameter to the function. It is a structure that contains the droppped object(s). 

7.3.4 If it is a list of items [line 18] then we execute the "if" code. Otherwise if it is a list of files [line 33] we execute the "else" code.

Essentially both branches are the same, except the "if" branch checks whether each list item is a kind of file. All other non-file list items are ignored. 

You can modify this code if you want to check for other kinds of things, like text strings. 

Each file object in the ev list is passed to the function processFile [line 29, 40 depending which "if" branch was followed] for processing.

7.4 processFile is run from dropHandler, for each file object dropped onto the runtime Story browser window. It is customised to my application needs, that wants audio files. You can totally rewrite this to satisfy your application needs.

7.4.1 The "file" parameter is a structured object. We look at the "file.type" to determine if this file is an audio file - it will look like "audio/mp4" for an mp4 audio file. The console.logs at the start of the function [lines 48-49] print the file.type and file.name values in the runtime browser debug window.

7.4.2 We "split" the file type string at the "audio/" characters [line 51]. Split produces an array, assigned to constant ftype. In the case of "audio/mp4", ftype[0] will become "audio" and ftype[1] will become "mp4". 

7.4.3 If we have at least 2 elements in the ftype array [line 52] then we know it is an audio file.

7.4.4 We now need to find out what kind of audio file. The "switch" statement [line 56] is a kind of generalised "if" statement. We can check each case.

7.4.5 If our file is an "mp3" [line 58] or an "x-m4a" [line 59] then we want it. Otherwise for all other kinds, we discard it [lines 85-89].

7.4.6 If we want the file [lines 60-83] we want to append its name to any existing string in the Storyline text input field. The idea is we might have a directory location already typed in, and wish to append this file name. 

7.4.7 We get the current value of the text input field from the Storyline variable "TrackNameValue", using GetPlayer().GetVar() in the usual way.

[7.4.8Note to Articulate in general - it would be great if we could have GetState as well as GetVar :-). But I think I understand that would incur overhead - unless the Story maintained a private variable to record the state of an object. ]

7.4.9 [lines 66-77] we manipulate the value of TrackNameValue as copied into variable iStr [line 64], to remove anything after any final (*nix '/' or windows '\\') path seperator. 

The idea is we might keep a directory name if we are dragging a number of audio files in turn. Rather than type in the path name each time. This is specific to my application. 

7.4.10 We append the droppped file name to the end of iStr [line 78].

7.4.11 And then we set the Story variable "TextEntry", which is the name of our input field, with the resultant path / filename. 

7.5 And that's what the Javascript does. Executes our drag and drop functionality.

 

How To Build and Use

H1. Publish to web. View Project. 

H2. Type something in. Drag and drop something. 

 

Q&A

Q1. Why does the dropped filename not include the full pathname?

A1.1 Windows Explorer, or any file explorer, is quite a complicated piece of application software. It knows what directory you display when you click on a file object. Our javascript doesn't have that functionality. What you drop is what you get.

A1.2 Also remember when you publish to Web, the Story will be run in a Player in a browser. More precisely a virtual machine within the browser. From here, there is no visibility of your computer's file system.

Q2. Can I drop other kinds of file?

A2. Yes. My code handles audio files because that is what I want for my application. You can process any kind of file. By modifying / rewriting processFile. See here: https://developer.mozilla.org/en-US/docs/Web/API/DataTransferItem/getAsFile 

Q3. Can I drop non-file things?

A3. Yes. Same answer as A2. And you will have to modify dropHandler too.

Q4. Is Javascript supported by Articulate?

A4. Articulate provide for Javascript through the trigger which allows you to execute it. But Articulate provides no support for your code. In writing Javascript you are on your own, or with the community here. Or Mr. Google. 

I have found several ways to "break" a runtime Story using Javascript. You will not get help with Javascript code; it exceeds the provision of Articulate support.

Q5. I am not a Javascript programmer, can you help me?

A5. No. Suggest you teach yourself the rudimentaries. Try: https://www.w3schools.com/js/ 

Tip: Don't give up, use console log a lot.

Q6. Is embedding Javascript in the way shown here shown the best approach?

A6. No. I have simplified the story file to focus on drag and drop. In my Storyline 'applications' (ones with a lot of Javascript) I place my low-level Java functions, such as those described in this article, in a separate file. This file is loaded dynamically at runtime through a web object. (I think I borrowed this idea from Chris or Mat, and made it robust.) The javascript embedded in the triggers manages the 'GUI' or Storyline slide stuff. 

Q7. How do I see the browser debug window?

A7. I use Chrome. Click on a browser window / tab. Press CTRL+SHIFT+J. Select "console" to see console logs.

Q8. Does this code work on other browsers / all browsers.

A8. The code used here should work on any fairly recent browser. It works on my Samsung phone for example. (Although drag and drop isn't very useful on my phone, it doesn't break.)

Q9. I tried dropping a file on your story and nothing happened!?

A9. My application only recognises .mp3 and .x-m4a audio files. Try displaying the browser debug window, and look at the console logs. You should see output with whatever you drag and drop, including rejects. Then work out what's going on from the javascript code. 

Q10. Can I publish to targets other than Web, including Review360?

A10. Yes. Web is just quickest for hacking. I mean development. 

 

Exercise

E1. Try modifying the javascript to accept an 'mp4' audio file. Hint: it's a one-liner, one case.