View source for DSpace/C2/Batch-import-of-items/English

'''Script : Batch import of Items'''

'''Author : Pankaj Patil'''

'''Keywords: SAFBuilder, Batch Import, Batch Import in DSpace, Item Submission, Dublin Core, Mapfile, Metadata Spreadsheet, Mapfile, Batch Job, Spoken Tutorial, Video Tutorial'''

{| border=1
 |- 
| align=center| '''Visual Cue'''
| align=center| '''Narration '''
|- 
|| Slide: Title
|| Welcome to this spoken tutorial on '''Batch import of Items.'''
|- 
|| Slide: Learning Objectives
|| In this tutorial, we will learn to
* Set up''' SAFBuilder'''
* Create''' '''a''' Simple Archive Format '''and
* Do a '''batch import''' of '''Items''' into a''' Collection'''
|- 
|| Slide: System requirements
|| This tutorial is recorded using
* '''Ubuntu Linux OS 18.04'''
* '''DSpace version 6.3'''
* '''Firefox web browser '''and
* '''Gedit Text Editor'''
However you may use any other '''web browser '''and '''text Editor '''of your choice.
|- 
|| Slide: Pre-requisites
|| To practice this tutorial, you should have
* A working '''internet''' connection
* Installed '''DSpace 6.3''' on your system
* '''Tomcat service''' should be running
|- 
|| Slide: Pre-requisites
|| 
* Have '''Administrator’s authority''' in '''DSpace '''and
* Created a '''Collection''' in the '''Community'''
* If not then please go through the prerequisite tutorials on this website.
|- 
|| Slide: Pre-requisites 
|| To follow this tutorial, you should have
* Knowledge of '''Library Science '''and
* Familiarity with '''Dublin Core metadata standards'''
|- 
|| Slide : Code files
|| 
* The''' '''files used in this tutorial are available in the '''Code Files '''link on this tutorial page.
* Please download and extract the files before practicing.
|- 
|| Slide: '''Batch Import Feature'''
||
* '''Batch Import '''feature facilitates the upload of a large number of '''Items''', simultaneously.
* '''DSpace administrator''' is '''authorized''' to use the '''Batch Import '''feature.
|- 
|| Slide: '''Batch Import'''
||
* '''Items '''uploaded using''' Batch Import''' are directly '''archived'''.
|- 
|| Slide:''' Batch Import Methodology'''
||
* Basically, '''metadata''' for each '''Item''' is entered in a '''spreadsheet''' along with a filename.
* Then we have to compress the '''spreadsheet''' and '''Item files''' into a '''zip''' file using '''SAFBuilder'''.
* Using '''administrator authority''', we have to upload the '''zip '''file.
|- 
|| Slide:''' Batch Import using SAFBuilder '''
||
* To import a '''batch''' of''' Items''', '''DSpace''' requires '''Items''' in the '''Simple Archive Format''' i.e. '''SAF.'''
* '''SAFBuilder''' is the tool that is used to create the '''Simple Archive Format.'''
|- 
|| Slide: Setting up '''SAFBuilder''' 
||
*  '''SAFBuilder '''is an add-on tool, which can be set up''' '''for any '''Ubuntu Linux user'''.
* The '''manual''' to set up '''SAFBuilder''' on '''Windows '''is provided as '''Additional Reading Material''' on this tutorial page.
|- 
|| Slide: '''Simple Archive Format'''
|| Let us see the structure for '''Simple Archive Format'''.
* The '''archive''' contains one '''subdirectory''' per''' Item'''.
* Each '''subdirectory''' contains # '''dublin_core.xml '''file i.e. the '''Item's metadata'''
* '''Files''' that come along with the '''Item '''and
* The '''Contents '''file, which consists of a list of filenames. 
|- 
|| Narration Only
|| Let us proceed to set up''' '''the '''SAFBuilder '''tool.
|- 
|| Press '''Ctrl+Alt+T '''keys
|| Open the '''terminal''' by pressing '''Ctrl + Alt + T''' keys simultaneously on the keyboard.
Ensure that you have '''root permissions''' to run the '''commands'''.
|- 
|| Only Narration
|| Here onwards please press the '''Enter''' key after typing each '''command'''.
|- 
|| Highlight user '''spoken'''
|| I will set up the '''SAFBuilder '''tool on the '''user spoken '''on my machine'''.'''
|- 
|| Only Narration
|| '''git, jdk '''and '''maven '''tools are used to download and '''compile''' '''SAFBuilder, '''respectively'''.'''
These are already installed during '''DSpace '''installation.
|- 
|| [Terminal]: Type
'''git clone https://github.com/DSpace-Labs/SAFBuilder.git'''
|| Type the following '''command''' to download '''SAFBuilder '''from '''DSpace-Labs repository''' in '''github.'''
The '''SAFBuilder '''download may take some time depending on your '''internet '''speed.


If the '''SAFBuilder''' download fails, then recheck your '''internet''' connection or try after some time.
'''SAFBuilder''' download is now completed on my machine.
|- 
|| [Terminal]: Type
'''cd SAFBuilder'''
|| Now type '''cd SAFBuilder '''to change the current directory to '''SAFBuilder '''directory'''.'''
|- 
|| [Terminal]: Type
'''./safbuilder.sh'''
|| Then type '''./safbuilder.sh''' to '''compile SAFBuilder.'''
'''Compilation''' of '''SAFBuilder''' has started and it may take some time to complete.
If '''SAFBuilder compilation''' fails, then recheck your '''internet''' connection or try after some time.
|- 
|| Narration only
|| The '''SAFBuilder compilation''' is now successful.
|- 
|| Narration only
|| Next, let us proceed to create a '''Simple Archive Format''' of different '''Items'''.
|- 
|| [Terminal]: Type
'''cd $HOME'''
|| Type '''cd $HOME, '''to change the current directory to '''Home''' directory.
|- 
|| [Terminal]: Type
'''mkdir''' '''ItemUpload'''
|| Now let us create a directory to store the '''metadata spreadsheet '''and all the files to be uploaded.
I will create a directory named '''ItemUpload.'''
To do so, type '''mkdir''' '''ItemUpload.'''
|- 
|| Narration only
|| I have downloaded an '''article''' and its '''metadata '''files in my '''Downloads '''folder.
These are provided in the '''Code Files '''link.
|- 
|| [Terminal]: Type
'''cd $HOME/Downloads'''
|| Using '''cd command''' we will switch to the '''Downloads '''directory.
|- 
|| [Terminal]: Type '''ls'''
|| Type '''ls''' to check the contents. 
|- 
|| Highlight '''Article1.pdf '''and '''Article2.pdf'''
|| For this demonstration, we will use '''Article1.pdf '''and '''Article2.pdf '''files for the '''batch upload'''.
|- 
|| [Terminal]: Type
'''cp Article1.pdf Article2.pdf $HOME/ItemUpload'''
|| So, copy the files '''Article1.pdf '''and '''Article2.pdf''' to '''ItemUpload '''directory with this '''command'''.
|- 
|| Narration only
|| Now, let us proceed to check the '''metadata spreadsheet''' file.
|- 
|| Text on screen: In Windows OS, select Open with Excel
|| Right-click on '''Article1-2metadata.csv''' and select '''Open with LibreOffice Calc''' as shown here.
|- 
|| Point to dialog box'''Text Import'''
|| The '''Text Import''' dialog box appears, which have some settings in which to import the text.
|- 
|| Click the button '''OK'''
|| Keep the default settings and click on the '''OK''' button'''.'''
|- 
|| Point to '''Article1-2metadata.csv'''
|| '''Article1-2metadata.csv '''opens as a '''spreadsheet'''.
|- 
|| Point to '''header '''row
|| Observe that the first row is the '''header''' row.
|- 
|| Point to columns of '''header '''row
|| Each column of the '''header '''row is a '''Dublin Core''' element.
It corresponds to each field in the '''Item Submission form.'''
|- 
|| Narration only
|| It is mandatory to use '''header names '''strictly as provided in this '''metadata spreadsheet'''.''' '''
|- 
|| Point to column '''filename'''
|| The first column is '''filename, '''which is used to write the name of the file to be uploaded.
|- 
|| Narration only
|| The next columns sequentially represent each field in the '''Item Submission form.'''
|- 
|| Point to '''dc.contributor.author'''
|| For example, the first field in '''Item submission form''' is '''Authors.'''
The corresponding column for '''Author '''in the '''spreadsheet''' is '''dc.contributor.author'''
|- 
|| Point to multiple columns of
'''dc.contributor.author''' 
|| '''Author '''is a multi-value field in '''Item submission form'''.
So, multiple columns are represented as '''dc.contributor.author''' 
|- 
|| Narration only
|| The number of columns can be added or removed from multi-value fields in the '''Item Submission form.'''
|- 
|| Narration only
|| Similarly, we have columns for other fields in the '''Item Submission form.'''
|- 
|| Highlight '''dc.title,dc.date.issued,dc.publisher,etc'''
|| Single-value fields in the '''Item Submission form '''are represented using a single column.
|- 
|| Highlight '''dc.identifier.ismn, dc.identifier.issn, dc.identifier.isbn'''
|| Multi-value fields in the '''Item Submission form '''are represented using multiple columns.
|- 
|| Point to row of '''Article1.pdf'''
|| Each row in the '''spreadsheet''' represents a separate '''Item.'''
|- 
|| Point to row of '''Article1.pdf '''and '''Article2.pdf'''
|| For each '''Item '''to be uploaded, write the filename and '''metadata '''as shown.
|- 
|| Point to '''Article1-2metadata.csv'''
|| The '''metadata spreadsheet''' should be saved in '''CSV format''' only.
|- 
|| Close the '''Calc'''
|| Close the '''Calc''' window now.
|- 
|| Switch back to terminal
|| Switch back to the '''terminal'''.
|- 
|| [Terminal]: Type
'''cp Article1-2metadata.csv $HOME/ItemUpload'''
|| Now, copy the '''metadata spreadsheet''' file to the '''ItemUpload '''directory'''.'''
To do so, type the '''command''' as shown here.
|- 
|| [Terminal] : Type
'''cd $HOME/ItemUpload'''
|| Now let us check the contents of the '''ItemUpload '''directory.
Using '''cd command,''' change current working directory to the '''ItemUpload '''directory'''.'''
|- 
|| [Terminal] : Type '''ls'''
|| Then type '''ls.'''
|- 
|| Highlight '''Article1.pdf, Article2.pdf, Article1-2metadata.csv'''
|| '''ItemUpload '''directory has the files to be uploaded and their '''metadata''' in a '''CSV''' file.
i.e. '''Article1.pdf, Article2.pdf '''and '''Article1-2metadata.csv'''
|- 
|| Narration only
|| Now, let us proceed to create a '''zip '''file in the '''SAF''' format..
|- 
|| [Terminal]: Type
'''cd $HOME/SAFBuilder'''
|| Type this '''command '''to change the present working directory to '''SAFBuilder'''.
|- 
|| [Terminal]: Type
'''./safbuilder.sh -c $HOME/ItemUpload/Article1-2metadata.csv -z'''
|| Type the next '''command '''as shown, to prepare a '''Simple Archive '''file.
The '''Simple Archive file '''creation is successful.
|- 
|| Narration only
|| The '''Simple Archive file '''in '''zip '''format will be created in the directory of the '''metadata spreadsheet'''.
In our case it is '''ItemUpload.'''
|- 
|| [Terminal] : Type
'''cd $HOME/ItemUpload'''
|| Type this''' command '''to change the current directory to '''ItemUpload '''directory'''.'''
|- 
|| [Terminal] : Type '''ls'''
|| Type the '''command ls''' to get a list of files in '''ItemUpload '''directory'''.'''
|- 
|| Highlight '''SimpleArchiveFormat.zip'''
|| '''SimpleArchiveFormat.zip '''is seen here.
|- 
|| Narration only
|| Now, let us proceed to upload the '''zip''' file for '''batch import''' in '''DSpace'''.
|- 
|| '''Web browser >> Address bar >> localhost:8080'''
|| Open the '''DSpace '''interface.
|- 
|| Log into '''DSpace '''with '''admin '''role
Email:''' dspace.u1@gmail.com '''
Password: '''u1pass'''
|| Login to '''DSpace '''with your '''administrator authority'''.
I will login with''' '''my '''administrator authority.'''
|- 
|| Click on the '''logged in''' tab
|| Click on the''' Logged in''' tab at the top right corner.
|- 
|| Select '''Administer'''
|| Select '''Administer '''from the drop-down.
|- 
|| Click on '''Content''' tab
|| Click on the '''Content''' tab in the '''Navigation bar'''.
|- 
|| Select '''Batch import'''
|| From the drop-down, select '''Batch import'''.
|- 
|| Point to '''Batch import'''
|| The '''Batch import '''page opens.
|- 
|| Point to '''Select the type of the input data'''
|| '''Select type of the input data, Simple Archive Format (zip file via upload) '''is selected by default.
|- 
|| click on '''Browse '''button
|| To upload the '''SAF zip '''file, click on the '''Browse '''button in the '''Select data file to upload '''field'''.'''
|- 
|| Point to '''File Upload'''
|| The '''File Upload '''dialog box''' '''opens up.
|- 
|| Select '''SimpleArchiveFormat.zip'''
|| Browse and select the file '''SimpleArchiveFormat.zip '''
|- 
|| Click '''Open '''button
|| Then, click on the '''Open '''button. 
|- 
|| Point to '''SimpleArchiveFormat.zip'''
|| On success, the name of the file''' '''is displayed next to the '''Browse '''button'''.'''
|- 
|| Click on '''Select Collection''' drop down
|| Click on '''Select the owning collection of the items''' drop-down.
|- 
|| Select the '''Articles Collection '''
|| From the list, select the '''Collection '''into which we want to upload the</span>''' Items.'''</span>
I will select '''Articles.'''
|- 
|| Point to '''Select other collections that the items will belong to'''
|| '''Select other collections that the items will belong to '''field appears above the '''Upload''' button.
For this demonstration, I’m not selecting any other '''Collection'''. 
|- 
|| Click '''Upload''' button
|| Click on the '''Upload''' button at the bottom of the page.
|- 
|| Point to notification
|| A message is displayed, “'''The job was taken over, an email will be sent as soon as it’s finished'''”.
|- 
|| Only Narration
|| We can see the progress of the '''batch import''' in the '''My DSpace''' page.
|- 
|| Click on '''My DSpace '''link
|| Click on '''My DSpace '''link'''.'''
|- 
|| Point to '''Batch imports'''
|| In the '''Batch Imports '''section we can see the '''timestamp''' of the job submission for the '''batch import.''' 
|- 
|| Point to '''success'''
|| Also we can see the '''success status''' for this '''batch import '''job.
|- 
|| click on '''show more '''link
|| To view more details, click on '''Show more '''link''' '''next to the '''timestamp''' of the job submission. 
|- 
|| Point to''' Items to be imported '''and '''Items imported '''
|| '''Items to be imported''' and '''Items imported''' along with the number of '''items''' is displayed.
|- 
|| click on the link '''show items'''
|| To view the imported '''Items,''' click on the link '''Show items '''next to the label '''Items imported.'''
|- 
|| Narration Only
|| '''Mapfile''' contains mapping of '''Items''' uploaded in the '''batch''' and their '''handle '''numbers.
|- 
|| Point to the '''Download mapfile''' button
|| Download the '''mapfile''' by clicking on the '''Download mapfile''' button below the label '''Items imported.''' 
|- 
|| Narration Only
|| We can also delete the '''Items''' that were uploaded using''' batch import'''. 
|- 
|| Point to '''Delete uploaded items & remove imports '''button
|| To do so, use the button '''Delete uploaded items & remove import, '''next to '''Download mapfile''' button.
|- 
|| Narration Only
|| For this demonstration, I will not be deleting the '''Items''' imported as a '''batch'''.
|- 
|| Narration only
|| Now, let us cross-verify the '''batch import''' of the '''Items''' in the '''Collection.'''
|- 
|| click the '''Browse''' tab
|| To do so, click the '''Browse''' tab in the '''Navigation bar'''.
|- 
|| Select '''Communities and Collections'''
|| Then click on '''Communities and Collections''' from the drop-down.
|- 
|| Select the '''Article Collection '''
|| Select the '''Articles Collection.'''
|- 
|| Point to '''Collection Home Page'''
|| The '''Collection Home Page '''appears.
|- 
|| Scroll down
|| Scroll down to locate the '''Items '''we imported in the '''Collection. '''
|- 
|| Point to '''Items'''
|| We can see that the '''Items''' submitted using '''batch import''' are successfully uploaded here.
|- 
|| Point to '''Items'''
|| Sometimes,''' Items '''uploaded in a '''batch''' may take some time to appear in the '''Collection.'''
|- 
|| Select the first '''Item'''
|| Select the first '''Item''' submitted using '''batch import.'''
|- 
|| Point to '''metadata '''and file
|| We can see the '''metadata''' of the '''Item''' and its file. 
|- 
|| Narration only
|| This means, we have successfully uploaded '''Items''' using '''SAFBuilder''' and '''Batch import'''.
|- 
|| Narration only
|| On success of the '''Batch Import '''job, an '''email''' notification is also sent to the '''Administrator.'''
Let us proceed to cross-verify the email notification.
|- 
|| Log into administrator’s email account 
Email:''' dspace.u1@gmail.com '''
Password: '''d$pace2019*'''
|| Log in to your '''administrator's email account'''.
This is my '''administrator's email account.'''
|- 
|| Point to mail 
'''DSpace - Batch import successfully completed'''
|| Here is the '''email''' with the subject '''DSpace - Batch import successfully completed'''
|- 
|| Narration Only
|| If the '''email''' is not seen in the '''Inbox''', then it could be in the '''SPAM''' folder.
Otherwise please recheck your''' internet connection'''.
|- 
|| Open a mail
|| Let us open the '''email'''.
|- 
|| Highlight '''mapfile '''path 
|| The email contains a '''mapfile path'''.
|- 
|| Narration Only
|| So now we have verified the email notification of successful completion of '''batch import'''.
|- 
|| Switch back to '''DSpace'''
|| Switch back to '''DSpace.'''
|- 
|| '''Logout '''from '''DSpace'''
|| Let us '''logout''' from the '''DSpace''' interface.
|- 
|| Only Narration
|| This brings us to the end of this tutorial.
Let us summarize.
|- 
|| Slide: Summary
|| In this tutorial we learnt to
* Set up''' SAFBuilder'''
* Create''' Simple Archive Format '''and
* Do a '''batch import''' of '''Items''' into a''' Collection'''
|- 
|| Slide: Assignment
|| As an assignment,
* Create a '''metadata spreadsheet''' for '''Article3.pdf''' and '''Article4.pdf'''
* Use '''metadata''' provided in the '''Code Files''' for '''Article3.pdf''' and '''Article4.pdf'''
* Upload '''Article3.pdf''' and '''Article4.pdf '''in '''Article Collection '''using '''batch import'''.
|- 
|| Slide : About Spoken Tutorial project
|| The video at the following link summarises the '''Spoken Tutorial project.'''
Please download and watch it.
|- 
|| Slide : Spoken Tutorial workshops
|| The '''Spoken Tutorial Project''' team conducts workshops and gives certificates.
For more details, please write to us.
|-
|| Slide: Forums
|| Please post your timed queries in this Forum.
|- 
|| Slide: Acknowledgement -I
|| '''Spoken Tutorial project''' is funded by MHRD, Government of India.
|- 
|| Slide: Acknowledgement -II
|| '''DSpace spoken tutorial''' series is funded by the National Virtual Library of India, Ministry of Culture, Government of India.
|- 
|| Narration only
|| This script and video for this tutorial was contributed by '''Pankaj Patil''' from''' IIT Bombay.'''
And this is '''Nancy Varkey '''signing off. Thanks for joining.
|-
|}