Nancyvarkey at 11:39, 23 September 2021

2021-09-23T11:39:46Z

Pankajpatil694: Created page with "'''Script :DSpace''' '''Statistical Report Configuration''' '''Author : Pankaj Patil''' '''Keywords: DSpace Statistics, Statistical Report Configuration, Generate''' '''st..."

2021-09-22T06:56:38Z

Created page with "'''Script :DSpace''' '''Statistical Report Configuration''' '''Author : Pankaj Patil''' '''Keywords: DSpace Statistics, Statistical Report Configuration, Generate''' '''st..."

New page

'''Script :DSpace''' '''Statistical Report Configuration'''

'''Author : Pankaj Patil'''

'''Keywords: DSpace Statistics, Statistical Report Configuration, Generate''' '''statistics, Cronjobs, Cron Task Dspace statistics, Crontab, Community strength, Collection Strength, Items count in Community, automate dspace statistics, spoken tutorial, video tutorial'''

{| border=1
|-
|| '''Visual Cue'''
|| '''Narration'''
|-
|| Slide: Title
|| Welcome to this Spoken Tutorial on '''DSpace''' '''Statistical Report Configuration.'''
|-
|| Slide: Learning Objectives
|| In this tutorial, we will learn to
* Generate a '''statistics report'''
* Schedule the generation of a '''statistics report'''
* Display the number of '''Item submissions''' in '''Communities''' and '''Collections'''
|-
|| Slide: System requirements
|| This tutorial is recorded using
* '''Ubuntu Linux OS 18.04'''
* '''DSpace version 6.3'''
* '''gedit text editor''' and
* '''Firefox web browser'''
However, you may use any other '''editor''' or '''browser''' of your choice.
|-
|| Slide: Pre-requisites
|| To practice this tutorial, you should have
* A working '''internet connection''' and
* '''Installed DSpace 6.3 '''on your system
* '''Tomcat service''' should be running
|-
|| Slide: Pre-requisites
||
* Have '''archived Items '''in the''' Collections '''and
* Have '''Administrator’s authority''' in '''DSpace Collection'''
* If not then please go through the prerequisite tutorials on this website.
|-
|| Slide : Code files
||
* The '''commands''' used in this tutorial are in a file in the '''Code Files '''link on this tutorial page
* Please download and extract the file before practicing.
|-
|| Switch to '''gedit''' text editor and open the file
|| I have opened the '''Code file''' in the '''gedit text editor''' on my machine.
And I will use the same file to copy-paste the '''commands''' during the demonstration.
You may use any other '''text editor '''of your choice.
The shortcut to paste on the '''terminal''' is '''Shift, Ctrl '''and '''V '''keys'''.'''
|-
|| Slide: '''DSpace statistics'''
|| '''DSpace '''offers a basic '''web statistics package''' and offers the following '''statistics'''.
* Information on '''page views'''
* '''Searches''' performed
* '''User logins '''and
* Errors and warnings
|-
|| Slide: '''DSpace statistics'''
||
* By default, '''DSpace''' doesn’t generate''' Statistics reports'''.
* '''Statistics reports''' are generated by '''running scripts'''.
|-
|| Slide: '''DSpace statistics'''
||
* '''Scripts''' generate '''statistics reports''' by '''parsing DSpace logs'''.
* '''Scripts''' can be '''run''' manually or as a scheduled job.
* '''Statistics reports''' can be accessed only by the '''DSpace Administrator'''
|-
|| Only Narration
|| Let us begin to generate '''Statistics reports''' using '''scripts'''.
Initially, let us ensure that by default, '''Statistics reports''' are not generated''' '''by''' DSpace'''.
|-
| | Open Web Browser
| | Open a '''web browser.'''
|-
| | Go to''' localhost:8080'''
| | Go to the '''DSpace '''interface.
|-
| | Log into '''DSpace '''as
dspace.u1@gmail.com
Password: u1pass
| | '''Log in''' to '''DSpace '''with '''Administrator’s authority.'''
|-
|| Click on '''Logged in''' tab
|| Then click on the '''Logged in''' tab at the top right corner.
|-
|| Select '''Administer'''
|| Select '''Administer''' from the drop-down.
|-
|| Point to '''Administration Tools'''
|| '''Administration Tools '''page opens up.
|-
|| Click on '''Statistics '''tab in the navigation bar
|| Click on the '''Statistics '''tab in the '''navigation bar'''.
|-
|| Point to '''There are currently no reports available for this service. Please check back later.'''
|| A message is displayed saying,
'''“There are currently no reports available for this service. Please check back later.”'''
|-
|| Only narration
|| So we have verified that by default, '''DSpace''' doesn’t generate''' Statistics reports'''.
|-
|| Only narration
|| Now, let us learn how to generate '''Statistics reports''' using '''scripts'''.
|-
|| Press '''Ctrl+Alt+T '''keys
|| Open the '''terminal''' by pressing '''Ctrl + Alt + T''' keys simultaneously on the keyboard.
Ensure that you have '''root permissions''' to '''run''' the '''commands'''.
|-
|| Only Narration
|| Here onwards, please remember to press the '''Enter''' key after typing each '''command'''.
|-
|| [Terminal] type:
'''sudo su - dspace'''
|| Now, we will '''log in''' as '''user dspace'''.
Type '''sudo su - dspace'''
If prompted, then type the '''password '''of the''' admin user '''to '''login'''.
|-
|| '''Slide: DSpace Statistics reports '''
|| '''DSpace''' offers facilities to generate the following types of '''reports'''.
* '''Initial Report''': By default, monthly report for each month from '''January 2005.'''
* Starting month and year are '''configurable''' in the '''dstat.cfg '''file.
|-
|| '''Slide: DSpace Statistics reports '''
||
* '''Monthly Report''': Monthly report for the current month
* '''General Report: '''Aggregated report of all the months
|-
|| Switch back to terminal
|| Switch back to the '''terminal'''.
|-
|| Only Narration
|| Let us '''parse''' the '''DSpace logs '''to prepare the '''data''' for the creation of '''initial reports'''.
|-
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-initial '''
|| To do so, type the '''command''' as shown.
If prompted, enter the '''root user password'''.
|-
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-monthly'''
|| Type the '''command''' as shown to prepare the '''data''' for the creation of '''monthly reports'''.
|-
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-general'''
|| Similarly, prepare the '''data''' for the creation of '''general reports''' using this '''command'''.
|-
|| Only Narration
|| Now, let us proceed to create actual '''Statistics reports''' using the '''data''' prepared in the previous steps.
|-
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-report-initial '''
|| Type the '''command''' as shown to generate '''Monthly Statistics''' '''reports''' from the starting month and year.
|-
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-report-monthly'''
|| Generate current month '''Statistics report''' using this '''command'''.
|-
|| [Terminal] type: '''sudo $HOME/dspace-install/bin/dspace stat-report-general'''
|| Similarly, generate a '''General Statistics report,''' with this '''command'''.
|-
|| Only Narration
|| Now, let us proceed to access and verify the generated '''statistics reports'''.
|-
|| Switch back to '''DSpace'''
|| Switch back to '''DSpace.'''
|-
|| Reload the '''statistics''' page
|| Reload the '''Statistics''' page using the '''Reload '''button of the '''browser'''.''' '''
|-
|| Only Narration
|| If your '''session''' has been '''logged out''', please '''login''' again.
|-
|| Only Narration
|| '''Statistics report''' is generated and displayed.
|-
|| Only Narration
|| By default, the '''General report''' is displayed.
The '''General report''' is aggregated from '''January 2005''' till date.
|-
|| Point to '''General Overview''', '''Archive Information, Items Viewed, All Actions Performed '''and Others
|| '''Statistics''' page shows these reports in different sections such as -
'''General Overview, Archive Information, Items Viewed'''
'''All Actions Performed '''and many more useful sections.
|-
|| Point to '''Monthly reports'''
|| '''Monthly reports''' are available from '''January 2005''' onwards.
|-
|| Only Narration
|| '''Monthly reports '''can be viewed by clicking on the respective link of the '''Monthly report.'''
|-
|| Only Narration
|| Let us view the '''monthly report''' for '''August 2020.'''
|-
|| Click on '''monthly '''report of '''08/2020 '''
|| To do so, click on the '''monthly report''' of '''08/2020.'''
|-
|| Scroll down and show report
|| '''Statistics report''' of all the activities in''' August 2020 '''is displayed.
|-
|| Only Narration
|| This is how '''Statistics reports''' are manually generated and verified in '''DSpace'''.
|-
|| Only Narration
|| Manual generation of '''reports''' is not a good practice, as it is subject to human errors.
It is possible to automate '''report '''generation at a scheduled time of the day, week or month.
We will create a '''Bash script''' to generate '''Monthly '''and '''General Statistics reports'''.
And, we will automate the execution of the '''Bash script''' using '''crontab'''.
|-
|| Only Narration
|| As '''initial reports''' are already created, we will not automate '''initial reports generation'''.
|-
|| Only Narration
|| Let us automate the generation of '''General Statistics report''' on a daily basis.
|-
|| Switch back to terminal
|| Switch to the '''terminal.'''
|-
|| [Terminal] type:
'''sudo nano general-report-generation.sh'''
|| Type this '''command''' to create and open the '''Bash '''file '''general-report-generation.sh '''in '''gedit.'''
You can do so in an '''editor''' of your choice, as well.
If prompted, enter the '''root user password'''.
|-
|| Point to '''general-report-generation.sh'''
|| An empty '''general-report-generation.sh '''file opens in the '''text editor'''.
|-
|| Type in text editor
'''/home/dspace/dspace-install/bin/dspace stat-general'''
'''/home/dspace/dspace-install/bin/dspace stat-report-general'''
|| Type the '''commands''' for '''log parsing''' and '''General''' '''report generation''' as shown here.
|-
|| Save and close the file
|| Save and close the file.
|-
|| [Terminal] type:
'''sudo chmod 777''' '''general-report-generation.sh'''
|| Set '''executable permissions''' to '''general-report-generation.sh '''file with this '''command'''.
If prompted, type the '''root user password'''.
|-
|| [Terminal] type:
'''sudo chown dspace:dspace''' '''general-report-generation.sh'''
|| Type this '''command''' to set the '''ownership''' of '''general-report-generation.sh '''file to '''dspace user'''.
|-
|| Only Narration
|| Now, let us schedule the execution of the '''general-report-generation.sh script''' using '''crontab.'''
|-
|| [Terminal] type: '''crontab -e'''
|| Type the '''command''' as shown to '''configure crontab jobs'''.
|-
|| Only Narration
|| If you are '''accessing crontab '''for the first time, the system will prompt to enter a choice of '''text editor'''.
|-
|| [Terminal] type: '''1'''
|| Type '''1''' to set '''nano editor '''as the preferred '''editor'''.
But you can set any other '''editor''' of your choice, as well.
|-
|| Point to default configuration
|| The default '''configurations''' for '''crontab '''are shown.
|-
|| Slide : '''Crontab '''in Linux
||
* The '''format''' of '''crontab '''is '''Minute Hour DayOfMonth Month DayOfWeek command '''
|-
|| Slide : '''Crontab '''in Linux
||
* The permitted '''values''' for '''fields''' in '''crontab''' are as shown in the table.
|-
|| Switch back to''' terminal'''
|| Switch back to''' '''the '''terminal.'''
|-
|| Enter the following configurations
'''30 1 * * * /bin/sh /home/dspace/general-report-generation.sh'''
|| We will schedule the generation of '''General Statistics report''', every day at '''1:30 AM.'''
To do so, type the '''command''' as shown at the end of the file.
|-
|| Highlight '''30 1'''
|| The value of the '''Minutes '''field is set to '''30''' and the value of the '''Hour '''field is set to '''1'''.
|-
|| Highlight '''* * *'''
|| To '''execute''' the '''script''' every day, fields for '''DayOfMonth Month DayOfWeek '''is set to '''*'''.
|-
|| Only Narration
|| You can schedule '''report''' generation time as per any time of your choice.
|-
|| Press '''Ctrl + o'''
|| Save the '''configuration''' by pressing '''Ctrl''' and '''O''' keys together.
|-
|| Press '''Enter.'''
|| Press '''Enter '''to confirm the save.
|-
|-
|| Press '''Ctrl + x'''
|| Close the '''configuration''' file by pressing together '''Ctrl''' and '''X''' keys.
|-
|| Point to '''crontab : installing new crontab'''
|| '''crontab: installing new crontab '''message is shown on the '''terminal'''.
|-
|| Only narration
|| That means, the '''execution''' of the '''general-report-generation.sh '''file is scheduled and automated.
'''General Statistics reports''' will be created every day at '''1:30 AM.'''
|-
|| Only Narration
|| Next, let us automate the generation of '''Monthly Statistics reports''' on the last day of the month.
|-
|| [Terminal] type:
'''sudo nano monthly-report-generation.sh'''
|| Type the '''command''' as shown to create and open the '''bash '''file '''monthly-report-generation.sh '''in '''gedit.'''
You can do so in an '''editor''' of your choice, as well.
If prompted, type the '''root user password'''.
|-
|| Point to '''monthly-report-generation.sh'''
|| An empty '''monthly-report-generation.sh '''file opens in the '''text editor'''.
|-
|| Type in text editor
'''/home/dspace/dspace-install/bin/dspace stat-monthly'''
'''/home/dspace/dspace-install/bin/dspace stat-report-monthly'''
|| Type the '''commands''' for '''log parsing''' and '''monthly report generation''' as shown here.
|-
|| Save and close the file
|| Save and close the file.
|-
|| [Terminal] type:
'''sudo chmod 777''' '''monthly-report-generation.sh'''
|| Set '''executable permissions''' to '''monthly-report-generation.sh '''file, with this '''command'''.
If '''prompted''', type the '''root user password'''.
|-
|| [Terminal] type:
'''sudo chown dspace:dspace''' '''monthly-report-generation.sh'''
|| Type this '''command''' to set the '''ownership''' of '''monthly-report-generation.sh '''file to '''dspace user'''.
|-
|| Only Narration
|| Now, let us schedule the execution of '''monthly-report-generation.sh script''' using '''crontab.'''
|-
|| [Terminal] type: '''crontab -e'''
|| Type the '''command''' as shown to '''configure crontab jobs'''.
|-
|| Enter the following configurations
'''55 23 * * * [ "$(date +\%d -d tomorrow)" = "01" ] && /bin/sh /home/dspace/monthly-report-generation.sh'''
|| We will schedule the generation of '''Monthly Statistics reports''' on the last day of every month.
To do so, type this '''command''' at the end of the file.
|-
|| Highlight '''[ "$(date +\%d -d tomorrow)" = "01" ]'''
|| This '''command''' is entered to identify the last day of the month.
|-
|| Press '''Ctrl + o'''
|| Save the '''configuration''' by pressing '''Ctrl''' and '''O''' keys together.
|-
|| Press '''Enter.'''
|| Press '''Enter '''to confirm the save.
|-
|-
|| Press '''Ctrl + x'''
|| Close the '''configuration''' file by pressing together '''Ctrl''' and '''X''' keys.
|-
|| Point to '''crontab : installing new crontab'''
|| '''crontab: installing new crontab '''message is shown on the '''terminal'''.
|-
|| Only narration
|| The '''execution''' of '''monthly-report-generation.sh '''file is now scheduled and automated.
So, '''Monthly''' '''Statistics reports''' will be generated on the last day of every month.
|-
|| Only narration
|| Now, let us further '''configure DSpace '''to display the number of '''submissions''' in '''Community''' and '''Collections.'''
|-
|| Switch back to''' DSpace'''
|| Switch back to''' DSpace.'''
|-
|| Click''' Home '''tab
|| Click on the '''Home tab''' in the '''navigation bar'''.
|-
|| Scroll and locate '''Communities in DSpace''' section
|| Scroll down and locate '''Communities in the DSpace''' section.
|-
|| Point to '''Communities '''
|| The''' Community’s '''names and '''logos''' are displayed.
|-
|| Only Narration
|| But, the number of '''Item submissions''' in the '''Community''' is not displayed.
Also, the number of '''Item submissions''' in the '''Collections''' is not displayed.
|-
|| Only Narration
|| '''dspace.cfg '''file has a '''configuration''' to display the number of '''Item submissions''' in '''Community '''and '''Collections'''.
|-
|| [Terminal] type:
'''sudo nano $HOME/dspace-6.3-src-release/dspace/config/dspace.cfg'''
|| Switch to the '''terminal.'''
Open the '''dspace.cfg '''file with this '''command'''.
If prompted, enter the '''root user password.'''
|-
|| Locate the section '''Settings for item count (strength) information'''
|| Locate the section '''Settings for item count (strength) information.'''
|-
|| Point to''' webui.strengths.show'''
|| By default, the value for '''webui.strengths.show '''is set to '''false'''.
|-
|| Set '''webui.strengths.show = true'''.
|| Set '''webui.strengths.show '''value to '''true'''.
|-
|| Save and close '''dspace.cfg''' file
|| Save and close the '''dspace.cfg''' file.
|-
|| [Terminal] type:
'''$HOME/Rebuild-DSpace'''
|| Let us '''rebuild DSpace''' to update these changes.
|-
|| [Terminal]''' '''Highlight '''BUILD SUCCESSFUL '''message
|| Now, '''DSpace rebuild''' is successful.
|-
|| Only Narration
|| So, let us proceed to verify the changes.
|-
|| Switch back to''' DSpace'''
|| Switch back to''' DSpace.'''
|-
|| Click''' Home '''tab
|| Click on the '''Home '''tab in the '''navigation bar''' to refresh the page.
|-
|| Point to '''Community names '''and''' strengths'''
|| Observe that the number of '''Item submissions''' in the '''Community''' is now displayed.
|-
|| Narration only
|| So, we have successfully '''configured DSpace.'''
It displays the number of '''submissions''' in the '''Community '''and''' Collections.'''
|-
|| Narration only
| style="color:#252525;" | With this, we have come to the end of this tutorial.
|-
|| Narration only
|| Let us summarize.
|-
|| Slide: Summary
|| In this tutorial, we learnt to
* Generate a '''Statistics report'''
* Schedule the generation of a '''Statistics report '''
* Display the number of '''Item submissions''' in '''Communities''' and '''Collections'''
|-
|| Slide: Assignment
|| As an assignment,
* Generate '''initial, Monthly''' and '''General Statistics reports'''
* '''Configure''' and display the number of '''Item submissions''' in '''Communities''' and '''Collections'''
|-
|| Slide : About Spoken Tutorial project
|| The video at the following link summarises the '''Spoken Tutorial project.'''
Please download and watch it.
|-
|| Slide : Spoken Tutorial workshops
|| The '''Spoken Tutorial Project''' team conducts workshops and gives certificates.
For more details, please write to us.
|-
|| Slide: Forums
|| Please post your timed queries in this Forum.
|-
|| Slide: Acknowledgement -I
|| '''Spoken Tutorial project''' is funded by the Ministry of Education (MoE), Government of India.
|-
|| Slide: Acknowledgement -II
|| '''DSpace spoken tutorial''' series is funded by the National Virtual Library of India, Ministry of Culture, Government of India.
|-
|| Narration only
|| This script and video for this tutorial was contributed by '''Pankaj Patil''' from''' IIT Bombay.'''
And this is '''Nancy Varkey '''signing off. Thanks for joining.
|-
|}

@@ Line 180: / Line 180: @@
 || Point to '''General Overview''', '''Archive Information, Items Viewed, All Actions Performed '''and Others
 || '''Statistics''' page shows these reports in different sections such as -
-'''General Overview, Archive Information, Items Viewed'''
+'''General Overview, Archive Information, Items Viewed, All Actions Performed '''and many more useful sections.
-'''All Actions Performed '''and many more useful sections.
 |-
 || Point to '''Monthly reports'''
@@ Line 204: / Line 203: @@
 || Manual generation of '''reports''' is not a good practice, as it is subject to human errors.
 It is possible to automate '''report '''generation at a scheduled time of the day, week or month.
 We will create a '''Bash script''' to generate '''Monthly '''and '''General Statistics reports'''.
 And, we will automate the execution of the '''Bash script''' using '''crontab'''.
 |-
@@ Line 243: / Line 244: @@
 |-
 || Only Narration
-|| Now, let us schedule the execution of the '''general-report-generation.sh script''' using '''crontab.'''
+|| Now, let us schedule the '''execution''' of the '''general-report-generation.sh script''' using '''crontab.'''
 |-
 || [Terminal] type: '''crontab -e'''
@@ Line 330: / Line 331: @@
 |-
 || Only Narration
-|| Now, let us schedule the execution of '''monthly-report-generation.sh script''' using '''crontab.'''
+|| Now, let us schedule the '''execution''' of '''monthly-report-generation.sh script''' using '''crontab.'''
 |-
 || [Terminal] type: '''crontab -e'''

DSpace/C3/DSpace-Statistical-Report-Configuration/English - Revision history

Nancyvarkey at 11:39, 23 September 2021

Pankajpatil694: Created page with "'''Script :DSpace''' '''Statistical Report Configuration''' '''Author : Pankaj Patil''' '''Keywords: DSpace Statistics, Statistical Report Configuration, Generate''' '''st..."