View source for DSpace/C3/DSpace-Statistical-Report-Configuration/English

'''Script :DSpace''' '''Statistical Report Configuration'''


'''Author : Pankaj Patil'''


'''Keywords: DSpace Statistics, Statistical Report Configuration, Generate''' '''statistics, Cronjobs, Cron Task Dspace statistics, Crontab, Community strength, Collection Strength, Items count in Community, automate dspace statistics, spoken tutorial, video tutorial'''


{| border=1
|- 
|| '''Visual Cue'''
|| '''Narration'''
|- 
|| Slide: Title
|| Welcome to this Spoken Tutorial on '''DSpace''' '''Statistical Report Configuration.'''
|- 
|| Slide: Learning Objectives
|| In this tutorial, we will learn to 
* Generate a '''statistics report'''
* Schedule the generation of a '''statistics report'''
* Display the number of '''Item submissions''' in '''Communities''' and '''Collections'''
|- 
|| Slide: System requirements
|| This tutorial is recorded using
* '''Ubuntu Linux OS 18.04'''
* '''DSpace version 6.3'''
* '''gedit text editor''' and
* '''Firefox web browser'''
However, you may use any other '''editor''' or '''browser''' of your choice.
|- 
|| Slide: Pre-requisites
|| To practice this tutorial, you should have
* A working '''internet connection''' and
* '''Installed DSpace 6.3 '''on your system
* '''Tomcat service''' should be running
|- 
|| Slide: Pre-requisites
|| 
* Have '''archived Items '''in the''' Collections '''and
* Have '''Administrator’s authority''' in '''DSpace Collection'''
* If not then please go through the prerequisite tutorials on this website.
|- 
|| Slide : Code files
|| 
* The '''commands''' used in this tutorial are in a file in the '''Code Files '''link on this tutorial page
* Please download and extract the file before practicing.
|- 
|| Switch to '''gedit''' text editor and open the file
|| I have opened the '''Code file''' in the '''gedit text editor''' on my machine.
And I will use the same file to copy-paste the '''commands''' during the demonstration.
You may use any other '''text editor '''of your choice.
The shortcut to paste on the '''terminal''' is '''Shift, Ctrl '''and '''V '''keys'''.'''
|- 
|| Slide: '''DSpace statistics'''
|| '''DSpace '''offers a basic '''web statistics package''' and offers the following '''statistics'''.
* Information on '''page views'''
* '''Searches''' performed
* '''User logins '''and
* Errors and warnings
|- 
|| Slide: '''DSpace statistics'''
|| 
* By default, '''DSpace''' doesn’t generate''' Statistics reports'''.
* '''Statistics reports''' are generated by '''running scripts'''.
|- 
|| Slide: '''DSpace statistics'''
|| 
* '''Scripts''' generate '''statistics reports''' by '''parsing DSpace logs'''.
* '''Scripts''' can be '''run''' manually or as a scheduled job.
* '''Statistics reports''' can be accessed only by the '''DSpace Administrator'''
|- 
|| Only Narration
|| Let us begin to generate '''Statistics reports''' using '''scripts'''.
Initially, let us ensure that by default, '''Statistics reports''' are not generated''' '''by''' DSpace'''.
|-
|  | Open Web Browser
|  | Open a '''web browser.'''
|-
|  | Go to''' localhost:8080'''
|  | Go to the '''DSpace '''interface.
|-
|  | Log into '''DSpace '''as 
dspace.u1@gmail.com
Password: u1pass
|  | '''Log in''' to '''DSpace '''with '''Administrator’s authority.'''
|- 
|| Click on '''Logged in''' tab
|| Then click on the '''Logged in''' tab at the top right corner.
|- 
|| Select '''Administer'''
|| Select '''Administer''' from the drop-down.
|- 
|| Point to '''Administration Tools'''
|| '''Administration Tools '''page opens up.
|- 
|| Click on '''Statistics '''tab in the navigation bar
|| Click on the '''Statistics '''tab in the '''navigation bar'''.
|- 
|| Point to '''There are currently no reports available for this service. Please check back later.'''
|| A message is displayed saying, 
'''“There are currently no reports available for this service. Please check back later.”'''
|- 
|| Only narration
|| So we have verified that by default, '''DSpace''' doesn’t generate''' Statistics reports'''.
|- 
|| Only narration
|| Now, let us learn how to generate '''Statistics reports''' using '''scripts'''.
|- 
|| Press '''Ctrl+Alt+T '''keys
|| Open the '''terminal''' by pressing '''Ctrl + Alt + T''' keys simultaneously on the keyboard.
Ensure that you have '''root permissions''' to '''run''' the '''commands'''.
|- 
|| Only Narration
|| Here onwards, please remember to press the '''Enter''' key after typing each '''command'''.
|- 
|| [Terminal] type: 
'''sudo su - dspace'''
|| Now, we will '''log in''' as '''user dspace'''.
Type '''sudo su - dspace'''
If prompted, then type the '''password '''of the''' admin user '''to '''login'''.
|- 
|| '''Slide: DSpace Statistics reports '''
|| '''DSpace''' offers facilities to generate the following types of '''reports'''.
* '''Initial Report''': By default, monthly report for each month from '''January 2005.'''
* Starting month and year are '''configurable''' in the '''dstat.cfg '''file.
|- 
|| '''Slide: DSpace Statistics reports '''
|| 
* '''Monthly Report''': Monthly report for the current month
* '''General Report: '''Aggregated report of all the months
|- 
|| Switch back to terminal
|| Switch back to the '''terminal'''.
|- 
|| Only Narration
|| Let us '''parse''' the '''DSpace logs '''to prepare the '''data''' for the creation of '''initial reports'''.
|- 
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-initial '''
|| To do so, type the '''command''' as shown.
If prompted, enter the '''root user password'''.
|- 
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-monthly'''
|| Type the '''command''' as shown to prepare the '''data''' for the creation of '''monthly reports'''.
|- 
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-general'''
|| Similarly, prepare the '''data''' for the creation of '''general reports''' using this '''command'''.
|- 
|| Only Narration
|| Now, let us proceed to create actual '''Statistics reports''' using the '''data''' prepared in the previous steps.
|- 
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-report-initial '''
|| Type the '''command''' as shown to generate '''Monthly Statistics''' '''reports''' from the starting month and year.
|- 
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-report-monthly'''
|| Generate current month '''Statistics report''' using this '''command'''.
|- 
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-report-general'''
|| Similarly, generate a '''General Statistics report,''' with this '''command'''.
|- 
|| Only Narration
|| Now, let us proceed to access and verify the generated '''statistics reports'''.
|- 
|| Switch back to '''DSpace'''
|| Switch back to '''DSpace.'''
|- 
|| Reload the '''statistics''' page
|| Reload the '''Statistics''' page using the '''Reload '''button of the '''browser'''.''' '''
|- 
|| Only Narration
|| If your '''session''' has been '''logged out''', please '''login''' again.
|- 
|| Only Narration
|| '''Statistics report''' is generated and displayed.
|- 
|| Only Narration
|| By default, the '''General report''' is displayed.
The '''General report''' is aggregated from '''January 2005''' till date.
|- 
|| Point to '''General Overview''', '''Archive Information, Items Viewed, All Actions Performed '''and Others
|| '''Statistics''' page shows these reports in different sections such as -
'''General Overview, Archive Information, Items Viewed'''
'''All Actions Performed '''and many more useful sections.
|- 
|| Point to '''Monthly reports'''
|| '''Monthly reports''' are available from '''January 2005''' onwards.
|- 
|| Only Narration
|| '''Monthly reports '''can be viewed by clicking on the respective link of the '''Monthly report.'''
|- 
|| Only Narration
|| Let us view the '''monthly report''' for '''August 2020.'''
|- 
|| Click on '''monthly '''report of '''08/2020 '''
|| To do so, click on the '''monthly report''' of '''08/2020.'''
|- 
|| Scroll down and show report
|| '''Statistics report''' of all the activities in''' August 2020 '''is displayed.
|- 
|| Only Narration
|| This is how '''Statistics reports''' are manually generated and verified in '''DSpace'''.
|- 
|| Only Narration
|| Manual generation of '''reports''' is not a good practice, as it is subject to human errors.
It is possible to automate '''report '''generation at a scheduled time of the day, week or month.
We will create a '''Bash script''' to generate '''Monthly '''and '''General Statistics reports'''.
And, we will automate the execution of the '''Bash script''' using '''crontab'''.
|- 
|| Only Narration
|| As '''initial reports''' are already created, we will not automate '''initial reports generation'''.
|- 
|| Only Narration
|| Let us automate the generation of '''General Statistics report''' on a daily basis.
|- 
|| Switch back to terminal
|| Switch to the '''terminal.'''
|- 
|| [Terminal] type: 
'''sudo nano general-report-generation.sh'''
|| Type this '''command''' to create and open the '''Bash '''file '''general-report-generation.sh '''in '''gedit.'''
You can do so in an '''editor''' of your choice, as well.
If prompted, enter the '''root user password'''.
|- 
|| Point to '''general-report-generation.sh'''
|| An empty '''general-report-generation.sh '''file opens in the '''text editor'''.
|- 
|| Type in text editor
'''/home/dspace/dspace-install/bin/dspace stat-general'''
'''/home/dspace/dspace-install/bin/dspace stat-report-general'''
|| Type the '''commands''' for '''log parsing''' and '''General''' '''report generation''' as shown here.
|- 
|| Save and close the file
|| Save and close the file.
|- 
|| [Terminal] type: 
'''sudo chmod 777''' '''general-report-generation.sh'''
|| Set '''executable permissions''' to '''general-report-generation.sh '''file with this '''command'''.
If prompted, type the '''root user password'''.
|- 
|| [Terminal] type: 
'''sudo chown dspace:dspace''' '''general-report-generation.sh'''
|| Type this '''command''' to set the '''ownership''' of '''general-report-generation.sh '''file to '''dspace user'''.
|- 
|| Only Narration
|| Now, let us schedule the execution of the '''general-report-generation.sh script''' using '''crontab.'''
|- 
|| [Terminal] type: '''crontab -e'''
|| Type the '''command''' as shown to '''configure crontab jobs'''.
|- 
|| Only Narration
|| If you are '''accessing crontab '''for the first time, the system will prompt to enter a choice of '''text editor'''.
|- 
|| [Terminal] type: '''1'''
|| Type '''1''' to set '''nano editor '''as the preferred '''editor'''.
But you can set any other '''editor''' of your choice, as well.
|- 
|| Point to default configuration
|| The default '''configurations''' for '''crontab '''are shown.
|- 
|| Slide : '''Crontab '''in Linux
|| 
* The '''format''' of '''crontab '''is '''Minute Hour DayOfMonth Month DayOfWeek command '''
|- 
|| Slide : '''Crontab '''in Linux
|| 
* The permitted '''values''' for '''fields''' in '''crontab''' are as shown in the table. 
|- 
|| Switch back to''' terminal'''
|| Switch back to''' '''the '''terminal.'''
|- 
|| Enter the following configurations
'''30 1 * * * /bin/sh /home/dspace/general-report-generation.sh'''
|| We will schedule the generation of '''General Statistics report''', every day at '''1:30 AM.'''
To do so, type the '''command''' as shown at the end of the file.
|- 
|| Highlight '''30 1'''
|| The value of the '''Minutes '''field is set to '''30''' and the value of the '''Hour '''field is set to '''1'''.
|- 
|| Highlight '''* * *'''
|| To '''execute''' the '''script''' every day, fields for '''DayOfMonth Month DayOfWeek '''is set to '''*'''.
|- 
|| Only Narration
|| You can schedule '''report''' generation time as per any time of your choice.
|- 
|| Press '''Ctrl + o'''
|| Save the '''configuration''' by pressing '''Ctrl''' and '''O''' keys together.
|- 
|| Press '''Enter.'''
|| Press '''Enter '''to confirm the save.
|-
|- 
|| Press '''Ctrl + x'''
|| Close the '''configuration''' file by pressing together '''Ctrl''' and '''X''' keys.
|- 
|| Point to '''crontab : installing new crontab'''
|| '''crontab: installing new crontab '''message is shown on the '''terminal'''.
|- 
|| Only narration
|| That means, the '''execution''' of the '''general-report-generation.sh '''file is scheduled and automated.
'''General Statistics reports''' will be created every day at '''1:30 AM.'''
|- 
|| Only Narration
|| Next, let us automate the generation of '''Monthly Statistics reports''' on the last day of the month.
|- 
|| [Terminal] type: 
'''sudo nano monthly-report-generation.sh'''
|| Type the '''command''' as shown to create and open the '''bash '''file '''monthly-report-generation.sh '''in '''gedit.'''
You can do so in an '''editor''' of your choice, as well.
If prompted, type the '''root user password'''.
|- 
|| Point to '''monthly-report-generation.sh'''
|| An empty '''monthly-report-generation.sh '''file opens in the '''text editor'''.
|- 
|| Type in text editor
'''/home/dspace/dspace-install/bin/dspace stat-monthly'''
'''/home/dspace/dspace-install/bin/dspace stat-report-monthly'''
|| Type the '''commands''' for '''log parsing''' and '''monthly report generation''' as shown here.
|- 
|| Save and close the file
|| Save and close the file.
|- 
|| [Terminal] type: 
'''sudo chmod 777''' '''monthly-report-generation.sh'''
|| Set '''executable permissions''' to '''monthly-report-generation.sh '''file, with this '''command'''.
If '''prompted''', type the '''root user password'''.
|- 
|| [Terminal] type: 
'''sudo chown dspace:dspace''' '''monthly-report-generation.sh'''
|| Type this '''command''' to set the '''ownership''' of '''monthly-report-generation.sh '''file to '''dspace user'''.
|- 
|| Only Narration
|| Now, let us schedule the execution of '''monthly-report-generation.sh script''' using '''crontab.'''
|- 
|| [Terminal] type: '''crontab -e'''
|| Type the '''command''' as shown to '''configure crontab jobs'''.
|- 
|| Enter the following configurations
'''55 23 * * * [ "$(date +\%d -d tomorrow)" = "01" ] && /bin/sh /home/dspace/monthly-report-generation.sh'''
|| We will schedule the generation of '''Monthly Statistics reports''' on the last day of every month.
To do so, type this '''command''' at the end of the file.
|- 
|| Highlight '''[ "$(date +\%d -d tomorrow)" = "01" ]'''
|| This '''command''' is entered to identify the last day of the month.
|- 
|| Press '''Ctrl + o'''
|| Save the '''configuration''' by pressing '''Ctrl''' and '''O''' keys together.
|- 
|| Press '''Enter.'''
|| Press '''Enter '''to confirm the save.
|-
|- 
|| Press '''Ctrl + x'''
|| Close the '''configuration''' file by pressing together '''Ctrl''' and '''X''' keys.
|- 
|| Point to '''crontab : installing new crontab'''
|| '''crontab: installing new crontab '''message is shown on the '''terminal'''.
|- 
|| Only narration
|| The '''execution''' of '''monthly-report-generation.sh '''file is now scheduled and automated.
So, '''Monthly''' '''Statistics reports''' will be generated on the last day of every month.
|- 
|| Only narration
|| Now, let us further '''configure DSpace '''to display the number of '''submissions''' in '''Community''' and '''Collections.'''
|- 
|| Switch back to''' DSpace'''
|| Switch back to''' DSpace.'''
|- 
|| Click''' Home '''tab
|| Click on the '''Home tab''' in the '''navigation bar'''.
|- 
|| Scroll and locate '''Communities in DSpace''' section
|| Scroll down and locate '''Communities in the DSpace''' section.
|- 
|| Point to '''Communities '''
|| The''' Community’s '''names and '''logos''' are displayed.
|- 
|| Only Narration
|| But, the number of '''Item submissions''' in the '''Community''' is not displayed.
Also, the number of '''Item submissions''' in the '''Collections''' is not displayed.
|- 
|| Only Narration
|| '''dspace.cfg '''file has a '''configuration''' to display the number of '''Item submissions''' in '''Community '''and '''Collections'''.
|- 
|| [Terminal] type:
'''sudo nano $HOME/dspace-6.3-src-release/dspace/config/dspace.cfg'''
|| Switch to the '''terminal.'''
Open the '''dspace.cfg '''file with this '''command'''.
If prompted, enter the '''root user password.'''
|- 
|| Locate the section '''Settings for item count (strength) information'''
|| Locate the section '''Settings for item count (strength) information.'''
|- 
|| Point to''' webui.strengths.show'''
|| By default, the value for '''webui.strengths.show '''is set to '''false'''.
|- 
|| Set '''webui.strengths.show = true'''.
|| Set '''webui.strengths.show '''value to '''true'''.
|- 
|| Save and close '''dspace.cfg''' file
|| Save and close the '''dspace.cfg''' file.
|- 
|| [Terminal] type:
'''$HOME/Rebuild-DSpace'''
|| Let us '''rebuild DSpace''' to update these changes.
|- 
|| [Terminal]''' '''Highlight '''BUILD SUCCESSFUL '''message
|| Now, '''DSpace rebuild''' is successful.
|- 
|| Only Narration
|| So, let us proceed to verify the changes.
|- 
|| Switch back to''' DSpace'''
|| Switch back to''' DSpace.'''
|- 
|| Click''' Home '''tab
|| Click on the '''Home '''tab in the '''navigation bar''' to refresh the page.
|- 
|| Point to '''Community names '''and''' strengths'''
|| Observe that the number of '''Item submissions''' in the '''Community''' is now displayed.
|- 
|| Narration only
|| So, we have successfully '''configured DSpace.'''
It displays the number of '''submissions''' in the '''Community '''and''' Collections.'''
|- 
|| Narration only
| style="color:#252525;" | With this, we have come to the end of this tutorial.
|- 
|| Narration only
|| Let us summarize.
|- 
|| Slide: Summary
|| In this tutorial, we learnt to
* Generate a '''Statistics report'''
* Schedule the generation of a '''Statistics report '''
* Display the number of '''Item submissions''' in '''Communities''' and '''Collections'''
|- 
|| Slide: Assignment
|| As an assignment,
* Generate '''initial, Monthly''' and '''General Statistics reports'''
* '''Configure''' and display the number of '''Item submissions''' in '''Communities''' and '''Collections'''
|- 
|| Slide : About Spoken Tutorial project
|| The video at the following link summarises the '''Spoken Tutorial project.'''
Please download and watch it.
|- 
|| Slide : Spoken Tutorial workshops
|| The '''Spoken Tutorial Project''' team conducts workshops and gives certificates.
For more details, please write to us.
|- 
|| Slide: Forums
|| Please post your timed queries in this Forum.
|- 
|| Slide: Acknowledgement -I
|| '''Spoken Tutorial project''' is funded by the Ministry of Education (MoE), Government of India.
|- 
|| Slide: Acknowledgement -II
|| '''DSpace spoken tutorial''' series is funded by the National Virtual Library of India, Ministry of Culture, Government of India.
|- 
|| Narration only
|| This script and video for this tutorial was contributed by '''Pankaj Patil''' from''' IIT Bombay.'''
And this is '''Nancy Varkey '''signing off. Thanks for joining.
|-
|}