Community code snippets – part 2

Not everything went as great as I had planned. But after all – yes, opening a popup from right mouse button in Monaco editor and loading GitHub repo along with the code is not only possible, but also quite easily achievable. Here’s a short video with the result:

As you can see, it’s not perfect. It can still be improved. But I’ve never been a person who finishes everything. I like to discover, to open new paths and to inspire others. I also enjoy doing things that have never been done before – and I believe this was one of these.

Preparation

For those of you who missed my introduction post, here’s a short recap:

There is a community code snippets repo on GitHub, moderated by ServiceNow employees
I thought, hey, how cool it would be to have these snippets available on the instance?
And I made them available on the instane
To be honest, my plan was different, but I had to improvise, since I couldn’t make the official GitHub API to work in GlideModal popup.

There are 3 prerequisites to this little project:

Own a GitHub account
Fork the SN community code snippets repo
Create a personal access token, as described in… GitHub API docs 🙂 Even though you won’t use it directly, doesn’t mean you don’t have to authenticate. It is possible to do it without authentication, but you will quickly hit rate limits.

Why can’t you use the GitHub API?

I’m not entirely sure. Honestly, it is working great when used in UI Page. To my surprise, even the ES modules loaded and you could use the ES6 syntax from within the scoped application. But when called from GlideModal on client script, all you could see was an error. Is it because of global client script calling scoped app UI Page? Or because of some CORS rules? Or how GlideModal includes the UI Page’s HTML? I’m not technical enough to explain

So what worked?

I was looking for a way to call the GitHub API without using their ES modules. One of the search queries I ran, somewhere on the bottom of the first page (yes, I was that desperate!) showed me the link staring with api.github.com. But wait a minute, isn’t it the same subdomain that I was hitting on the UI Page earlier? And if it’s displayed on the web serach results, it means that I could do a GET request? Visit this link and find out for yourself:

It’s exactly that, you can see the array of objects representing the contents of the main repository directory. By looking at the elements, you can quickly notice the attributes like name, download_url, path and content.

Great, but how do I read it from client script? I’ve been using the Internet long enough to know the $.get call. Unfortunatelly, it didn’t work. Probably jQuery was not loaded or something, I don’t know. I don’t like giving up until everything is confirmed to fail. So I went for the last resort, the good, old XMLHttpRequest. I remember reading some old code back at my first job that was using this. It’s a reliable API that simply has to work, it’s part of core JavaScript.

And in fact, that was all I needed:

Be able to GET repo contents based on URL
Be able to make a GET request from vanilla JavaScript

Sounds like a way ahead!

With this, and the fact that I know how to add a menu item to the right mouse button in Monaco editor, I could make a first draft of the script:

function onLoad() {
	addAfterPageLoadedEvent(
		function() {
			var myEd = GlideEditorMonaco.getAll()[0];
			
			if (myEd) {
				myEd.editor.addAction({
					'contextMenuGroupId': '10_monaco_ccs',
					'contextMenuOrder': 1,
					id: 'ccs_monaco_editor',
					label: 'Load communicty code snippets',
					run: function() {
						var root = document.createElement('div');
						loadRepoContent(root, '', function(modalHTML){
							var gm = new GlideModal(null, false, 1024);
							gm.setTitle('Community Code Snippets browser');
							gm.renderWithContent(modalHTML);
						}, 0);
					}
				});
			}
		}
	);
}

Why the callback? you may ask. That’s because of how I build my HTML. As you know, the GlideModal can also accept HTML, not only UI Page. This is what I do – in the loadRepoContent I recursively build HTML structure, add styling and event handlers and finally pass the result, the HTML element to the GlideModal.

Because of that, the loadRepoContent looks messy with a lot of text, but let’s analyse is, section by section

Repo loader step 1 – authentication

As I said, authentication is important if you want to use your repo browser as long as possible.

var baseURL = 'https://api.github.com/repos/{user_name}/{repo_name}/contents';
var authToken = 'Bearer SUPER_SECRET_PERSONAL_ACCESS_TOKEN';
var xhr = new XMLHttpRequest();
xhr.open('GET', baseURL + path, true);
xhr.setRequestHeader('Authorization', authToken);

Yes, make sure you add Bearer in the beginning of your Authorization header. And also remember that any call to the GitHub API should have the same Authorization header – it will be used later on.

Repo loader step 2 – build the list

Because it is a repo browser, the most natural way of presenting the data seems to be the list or some fancy tree views. When working on styles with pure JavaScript there are some limitations (you could probably make it prettier if you decided to implement it in the UI Page – but I don’t like Jelly). First step is to retrieve the data and build a structure of root-level nodes:

xhr.onreadystatechange = function() {
	if (xhr.readyState === XMLHttpRequest.DONE) {
		var status = xhr.status;
		if (status === 0 || (status >= 200 && status < 400)) {
			contents = JSON.parse(xhr.response);
			var contentList = document.createElement('ul');
			contents.forEach(function(content){
				var node = document.createElement('li');
				node.innerText = content.name;
				node.onclick = loadRepoContent(node, '/' + content.path);
				contentList.appendChild(node);
			});
		}
	}
}

Pretty basic stuff – once you have the data, create some elements, add text and click event handler. But it’s a trap! This code contains a serious bug. When you build your tree like that, it will continue calling the API until you hit your rate limit. If you are more familiar with JavaScript than me, you probably see it now – on line 10 I want to use some weird shortcut to handle the click event. But I just keep calling the loadRepoContent method. Yes, stupid mistake. Remember to learn on them! And use addEventListener('click', function(){}); to handle click events.

Repo loader step 3 – proper click handling and styling

Some finishing touches for the repo browser – we still don’t display the file contents:

xhr.onreadystatechange = function() {
	if (xhr.readyState === XMLHttpRequest.DONE) {
		var status = xhr.status;
		if (status === 0 || (status >= 200 && status < 400)) {
			contents = JSON.parse(xhr.response);
			var contentList = document.createElement('ul');
			contentList.style.listStyleType = 'none';
			contentList.style.margin = level * 3;
			contentList.style.padding = level * 3;
	
			contents.forEach(function(content){
				var node = document.createElement('li');
				var span = document.createElement('span');
				span.style.cursor = 'pointer';
				span.innerText = content.name;
				span.addEventListener('click', function(event){
					if (!this.nextElementSibling)
						loadRepoContent(node, '/' + content.path, null, level + 1);
					else
						this.parentNode.removeChild(this.nextElementSibling);
				});
				node.appendChild(span);
				contentList.appendChild(node);
			}
		}
	}
}

This is much better. It not only handles the click event properly, but also removes the DOM if you click on a node in order to hide it. It also doesn’t duplicate the child item creation when you click one of them (can you guess why? I’ll leave it as an exercise for you 😉 )

We still don’t know how to handle actual file display. Currently we are only traversing the path of the repo, up and down. But at the end of the path, we have files, with actual content. This is where we will have to make a second GET request – because the file have different URL.

Repo loader step 4 – load the file

If you visit this URL, you will notice the structure of the leaf node:

We have a size <> 0 and download_url <> 0. But we don’t want to download, we want to stay within the api.github.com subdomain and GET it. The url attribute will give us another object:

We have the content! But it looks like it’s encrypted? Now let’s go back to the GitHub API, it has all the hard answers. When you read about getting repository contents, although not clearly and immediately visible, it is written that the content is encoded in base64:

Great, so we have everything!

if (content.download_url) {
	var nodeForFile = document.createElement('li');
	var spanForFile = document.createElement('span');
	var details = document.createElement('details');
	var summary = document.createElement('summary');
	var contentXHR = new XMLHttpRequest();
	spanForFile.style.cursor = 'pointer';
	contentXHR.open('GET', content.url, true);
	contentXHR.setRequestHeader('Authorization', authToken);
	contentXHR.onreadystatechange = function() {
		if (contentXHR.readyState === XMLHttpRequest.DONE) {
			var contentStatus = contentXHR.status;
			if (contentStatus === 0 || (contentStatus >= 200 && contentStatus < 400)) {
				var textNode = document.createElement('pre');
				textNode.innerText = atob(JSON.parse(contentXHR.response).content);
				summary.innerText = JSON.parse(contentXHR.response).name;
				summary.style.fontStyle = 'italic';
				details.appendChild(summary);
				details.appendChild(textNode);
				spanForFile.appendChild(details);
				nodeForFile.appendChild(spanForFile);
				contentList.appendChild(nodeForFile);
			}
		}
	};
	contentXHR.send(null);
}

I chose pre tag to display the file contents. and remember about atob to decode it!

Entire code

Here’s a whole client script function, for your reference. You may of course go your own way, and I encourage you to do so. The code below for example doesn’t handle the images in the repository. Or generates basic tree view, nothing fancy. Or doesn’t have any icons to copy the code into the script field. Or… There are many ways to improve it!

function loadRepoContent(rootHTML, path, callback, level) {
	var contents = [];
	var baseURL = 'https://api.github.com/repos/{user_name}/{repo_name}/contents';
	var authToken = 'Bearer SUPER_SECRET_PERSONAL_ACCESS_TOKEN';
	var xhr = new XMLHttpRequest();
	xhr.open('GET', baseURL + path, true);
	xhr.setRequestHeader('Authorization', authToken);
	xhr.onreadystatechange = function() {
		if (xhr.readyState === XMLHttpRequest.DONE) {
			var status = xhr.status;
			if (status === 0 || (status >= 200 && status < 400)) {
				contents = JSON.parse(xhr.response);
				var contentList = document.createElement('ul');
				contentList.style.listStyleType = 'none';
				contentList.style.margin = level * 3;
				contentList.style.padding = level * 3;
				
				contents.forEach(function(content){
					if (content.download_url) {
						var nodeForFile = document.createElement('li');
						var spanForFile = document.createElement('span');
						var details = document.createElement('details');
						var summary = document.createElement('summary');
						var contentXHR = new XMLHttpRequest();
						spanForFile.style.cursor = 'pointer';
						contentXHR.open('GET', content.url, true);
						contentXHR.setRequestHeader('Authorization', authToken);
						contentXHR.onreadystatechange = function() {
							if (contentXHR.readyState === XMLHttpRequest.DONE) {
								var contentStatus = contentXHR.status;
								if (contentStatus === 0 || (contentStatus >= 200 && contentStatus < 400)) {
									var textNode = document.createElement('pre');
									textNode.innerText = atob(JSON.parse(contentXHR.response).content);
									summary.innerText = JSON.parse(contentXHR.response).name;
									summary.style.fontStyle = 'italic';
									details.appendChild(summary);
									details.appendChild(textNode);
									spanForFile.appendChild(details);
									nodeForFile.appendChild(spanForFile);
									contentList.appendChild(nodeForFile);
								}
							}
						};
						contentXHR.send(null);
					} else {
						var node = document.createElement('li');
						var span = document.createElement('span');
						span.style.cursor = 'pointer';
						span.innerText = content.name;
						span.addEventListener('click', function(event){
							if (!this.nextElementSibling)
								loadRepoContent(node, '/' + content.path, null, level + 1);
							else
								this.parentNode.removeChild(this.nextElementSibling);
						});
						node.appendChild(span);
						contentList.appendChild(node);
					}
				});
				rootHTML.appendChild(contentList);

				if (callback) callback(rootHTML);
			}
		}
	};

	xhr.send(null);
}

function onLoad() {
	addAfterPageLoadedEvent(
		function() {
			var myEd = GlideEditorMonaco.getAll()[0];
			
			if (myEd) {
				myEd.editor.addAction({
					'contextMenuGroupId': '10_monaco_ccs',
					'contextMenuOrder': 1,
					id: 'ccs_monaco_editor',
					label: 'Load communicty code snippets',
					run: function() {
						var root = document.createElement('div');
						loadRepoContent(root, '', function(modalHTML){
							var gm = new GlideModal(null, false, 1024);
							gm.setTitle('Community Code Snippets browser');
							gm.renderWithContent(modalHTML);
						}, 0);
					}
				});
			}
		}
	);
}

Conclusion

It’s always good to find a solution to an idea that’s been roaming in your head for a while. Even if at first you don’t succeed, don’t give up and find another way.