Create Translator for Zotero Connector, to scrape citation data from Sandpoints Journal Entries

anybody commented

2023-06-28 14:30:45 +02:00

Member

Translators for Zotero Connector (Zotero browser toolbar which grabs citation information from websites), are programmed in Javascript, via the Scaffold IDE included in Zotero Tools/Developer/Translator Editor.

It seems it's not difficult to create one, nevertheless we should think of a solution that is forward compatible for future Sandpoints projects.

Zotero Translators - The missing manual
https://www.zotero.org/support/dev/translators_reference_guide
Zotero guide for web translators
https://www.zotero.org/support/dev/translators/coding#web_translators
Guide for writing and testing a translator with Scaffold in Zotero:
https://www.zotero.org/support/dev/translators/scaffold

Required metadata for Journal articles:
https://aurimasv.github.io/z2csl/typeMap.xml#map-journalArticle

Translators for Zotero Connector (Zotero browser toolbar which grabs citation information from websites), are programmed in Javascript, via the Scaffold IDE included in Zotero Tools/Developer/Translator Editor. It seems it's not difficult to create one, nevertheless we should think of a solution that is forward compatible for future Sandpoints projects. Zotero Translators - The missing manual https://www.zotero.org/support/dev/translators_reference_guide Zotero guide for web translators https://www.zotero.org/support/dev/translators/coding#web_translators Guide for writing and testing a translator with Scaffold in Zotero: https://www.zotero.org/support/dev/translators/scaffold Required metadata for Journal articles: https://aurimasv.github.io/z2csl/typeMap.xml#map-journalArticle

anybody commented

2023-06-28 18:34:37 +02:00

Author

Member

I put together a Zotero Translator that half-works for Dotawo (for scraping citation data out of articles). The Javascript file is attached.

It only works for Single entries, but it can be made to scrape multiple entries from the Sitemap or individual Issues.

Notes:

If we fix it we can do a pull request to the Zotero translator repo.
I believe we should be able to make a generative version of this, to work on all Sandpoints projects.

Issues:

There is no publication date for Dotawo issues or articles!!!
Volume does not seem to apply

Successfully Scraped (automatically):

Authors
Tags
Title
URL
Website snapshot
Accessed date
Library catalogue

Dynamically scraped:

Abstract
Issue

Manually added:

Archive : item.archive="Sandpoints.org";
Publication : item.publicationTitle="Dotawo";

I put together a Zotero Translator that half-works for Dotawo (for scraping citation data out of articles). The Javascript file is attached. It only works for Single entries, but it can be made to scrape multiple entries from the Sitemap or individual Issues. Notes: - If we fix it we can do a pull request to the Zotero translator repo. - I believe we should be able to make a generative version of this, to work on all Sandpoints projects. Issues: - There is **no publication date** for Dotawo issues or articles!!! - Volume does not seem to apply Successfully Scraped (automatically): - Authors - Tags - Title - URL - Website snapshot - Accessed date - Library catalogue Dynamically scraped: - Abstract - Issue Manually added: - Archive : item.archive="Sandpoints.org"; - Publication : item.publicationTitle="Dotawo";

Dotawo .js

6.6 KiB

anybody commented

2023-06-29 11:41:57 +02:00

Author

Member

Alright, it seems I got the translator to work for:

single articles
issues
Sitemap

Citation info works well, with two issues:

the only field missing is the publication date, Dotawo doesn't appear to have one
Zotero saves an HTML snapshot of the page locally. In the case of downloading citations from the Sitemap or an issue, the offline page saved is the Sitemap or the Issue page, not the article page (FIX).

Marcell if we check and polish this, we can push to Zotero Translator repo. Javascript attached

Edit/install translator:

Open Zotero Tools/Developer/ Translator Editor
Set a folder, and place in that folder Dotawo.js.
Open Dotawo.js with the Scaffold IDE (which should pop up automatically).
Go to Browser tab, paste and enter a URL to test-scrape.
Press the Magnifying lens icon to detect
Press the Lightning icon to execute the translator on the URL.
If you are happy with that press save to Zotero.
Then on your browser right click on Zotero connector icon, and click Update Translators. Any code change, and save to Zotero will require to Update Translators.

Only install translator to your Zotero:

Open Zotero,
go to Preferences / Advanced / Files and Folders / Show Data Directory.
Then paste Dotawo.js in the translators folder.
On your browser right click on the Zotero connector icon, and click Update Translators.

Alright, it seems I got the translator to work for: - single articles - issues - Sitemap Citation info works well, with two issues: - the only field missing is the publication date, Dotawo doesn't appear to have one - Zotero saves an HTML snapshot of the page locally. In the case of downloading citations from the Sitemap or an issue, the offline page saved is the Sitemap or the Issue page, not the article page (FIX). Marcell if we check and polish this, we can push to Zotero Translator repo. Javascript attached Edit/install translator: - Open Zotero `Tools/Developer/ Translator Editor` - Set a folder, and place in that folder Dotawo.js. - Open Dotawo.js with the Scaffold IDE (which should pop up automatically). - Go to `Browser` tab, paste and enter a URL to test-scrape. - Press the `Magnifying lens` icon to detect - Press the `Lightning` icon to execute the translator on the URL. - If you are happy with that press save to Zotero. - Then on your browser right click on Zotero connector icon, and click `Update Translators`. Any code change, and `save to Zotero` will require to `Update Translators`. Only install translator to your Zotero: - Open Zotero, - go to Preferences / Advanced / Files and Folders / Show Data Directory. - Then paste Dotawo.js in the translators folder. - On your browser right click on the Zotero connector icon, and click Update Translators. ![image](/attachments/e9d2f3a0-4820-4a1d-bf10-514b24e633fa) ![image](/attachments/2e0f2ee4-4393-4f53-9fc4-2da53795d5d5) ![image](/attachments/6ec6532b-7fb8-4cc3-b0b6-d6412e8ce112) ![image](/attachments/eb69b32f-a634-4b26-98e3-e72933175983)

image.png

141 KiB

image.png

133 KiB

image.png

32 KiB

image.png

18 KiB

Dotawo .js

7.7 KiB

Rows
Columns

Create Translator for Zotero Connector, to scrape citation data from Sandpoints Journal Entries #11