mirrors/calibre-web
1
0
Fork 0
mirror of https://github.com/janeczku/calibre-web synced 2024-11-25 02:57:22 +00:00
Code Issues Releases Wiki Activity

Merge branch 'master' of github.com:GarckaMan/calibre-web into long_strip_cbrreader

Browse Source
This commit is contained in:
GarcaMan 2023-02-11 03:25:31 +00:00
commit c4326c9495
377 changed files with 144962 additions and 94732 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitattributes vendored
View File

@ -1,4 +1,5 @@
constants.py ident export-subst constants.py ident export-subst
/test export-ignore /test export-ignore
/library export-ignore
cps/static/css/libs/* linguist-vendored cps/static/css/libs/* linguist-vendored
cps/static/js/libs/* linguist-vendored cps/static/js/libs/* linguist-vendored

1
.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1 @@
custom: ["https://PayPal.Me/calibreweb",]

2
.gitignore vendored
View File

@ -23,10 +23,12 @@ vendor/
# calibre-web # calibre-web
*.db *.db
*.log *.log
cps/cache
.idea/ .idea/
*.bak *.bak
*.log.* *.log.*
.key
settings.yaml settings.yaml
gdrive_credentials gdrive_credentials

4
CONTRIBUTING.md
View File

@ -26,9 +26,9 @@ The Calibre-Web documentation is hosted in the Github [Wiki](https://github.com/
Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Instead, please write an email to "ozzie.fernandez.isaacs@googlemail.com". Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Instead, please write an email to "ozzie.fernandez.isaacs@googlemail.com".
Ensure the ***bug was not already reported** by searching on GitHub under [Issues](https://github.com/janeczku/calibre-web/issues). Please also check if a solution for your problem can be found in the [wiki](https://github.com/janeczku/calibre-web/wiki). Ensure the **bug was not already reported** by searching on GitHub under [Issues](https://github.com/janeczku/calibre-web/issues). Please also check if a solution for your problem can be found in the [wiki](https://github.com/janeczku/calibre-web/wiki).
If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=bug_report.md&title=). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue. If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new/choose). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue.
### **Feature Request** ### **Feature Request**

40
README.md Normal file → Executable file
View File

@ -1,6 +1,6 @@
# About # About
Calibre-Web is a web app providing a clean interface for browsing, reading and downloading eBooks using an existing [Calibre](https://calibre-ebook.com) database. Calibre-Web is a web app providing a clean interface for browsing, reading and downloading eBooks using a valid [Calibre](https://calibre-ebook.com) database.
[![GitHub License](https://img.shields.io/github/license/janeczku/calibre-web?style=flat-square)](https://github.com/janeczku/calibre-web/blob/master/LICENSE) [![GitHub License](https://img.shields.io/github/license/janeczku/calibre-web?style=flat-square)](https://github.com/janeczku/calibre-web/blob/master/LICENSE)
[![GitHub commit activity](https://img.shields.io/github/commit-activity/w/janeczku/calibre-web?logo=github&style=flat-square&label=commits)]() [![GitHub commit activity](https://img.shields.io/github/commit-activity/w/janeczku/calibre-web?logo=github&style=flat-square&label=commits)]()
@ -19,15 +19,16 @@ Calibre-Web is a web app providing a clean interface for browsing, reading and d
- full graphical setup - full graphical setup
- User management with fine-grained per-user permissions - User management with fine-grained per-user permissions
- Admin interface - Admin interface
- User Interface in brazilian, czech, dutch, english, finnish, french, german, greek, hungarian, italian, japanese, khmer, polish, russian, simplified and traditional chinese, spanish, swedish, turkish, ukrainian - User Interface in brazilian, czech, dutch, english, finnish, french, galician, german, greek, hungarian, indonesian, italian, japanese, khmer, korean, norwegian, polish, russian, simplified and traditional chinese, spanish, swedish, turkish, ukrainian, vietnamese
- OPDS feed for eBook reader apps - OPDS feed for eBook reader apps
- Filter and search by titles, authors, tags, series and language - Filter and search by titles, authors, tags, series, book format and language
- Create a custom book collection (shelves) - Create a custom book collection (shelves)
- Support for editing eBook metadata and deleting eBooks from Calibre library - Support for editing eBook metadata and deleting eBooks from Calibre library
- Support for downloading eBook metadata from various sources, sources can be extended via external plugins
- Support for converting eBooks through Calibre binaries - Support for converting eBooks through Calibre binaries
- Restrict eBook download to logged-in users - Restrict eBook download to logged-in users
- Support for public user registration - Support for public user registration
- Send eBooks to Kindle devices with the click of a button - Send eBooks to E-Readers with the click of a button
- Sync your Kobo devices through Calibre-Web with your Calibre library - Sync your Kobo devices through Calibre-Web with your Calibre library
- Support for reading eBooks directly in the browser (.txt, .epub, .pdf, .cbr, .cbt, .cbz, .djvu) - Support for reading eBooks directly in the browser (.txt, .epub, .pdf, .cbr, .cbt, .cbz, .djvu)
- Upload new books in many formats, including audio formats (.mp3, .m4a, .m4b) - Upload new books in many formats, including audio formats (.mp3, .m4a, .m4b)
@ -40,23 +41,21 @@ Calibre-Web is a web app providing a clean interface for browsing, reading and d
## Installation ## Installation
#### Installation via pip (recommended) #### Installation via pip (recommended)
1. Install calibre web via pip with the command `pip install calibreweb` (Depending on your OS and or distro the command could also be `pip3`). 1. To avoid problems with already installed python dependencies, it's recommended to create a virtual environment for Calibre-Web
2. Optional features can also be installed via pip, please refer to [this page](https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-Windows) for details 2. Install Calibre-Web via pip with the command `pip install calibreweb` (Depending on your OS and or distro the command could also be `pip3`).
3. Calibre-Web can be started afterwards by typing `cps` or `python3 -m cps` 3. Optional features can also be installed via pip, please refer to [this page](https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-and-Windows) for details
4. Calibre-Web can be started afterwards by typing `cps`
#### Manual installation In the Wiki there are also examples for: a [manual installation](https://github.com/janeczku/calibre-web/wiki/Manual-installation), [installation on Linux Mint](https://github.com/janeczku/calibre-web/wiki/How-To:Install-Calibre-Web-in-Linux-Mint-19-or-20), [installation on a Cloud Provider](https://github.com/janeczku/calibre-web/wiki/How-To:-Install-Calibre-Web-on-a-Cloud-Provider).
1. Install dependencies by running `pip3 install --target vendor -r requirements.txt` (python3.x). Alternativly set up a python virtual environment.
2. Execute the command: `python3 cps.py` (or `nohup python3 cps.py` - recommended if you want to exit the terminal window)
Issues with Ubuntu:
Please note that running the above install command can fail on some versions of Ubuntu, saying `"can't combine user with prefix"`. This is a [known bug](https://github.com/pypa/pip/issues/3826) and can be remedied by using the command `pip install --system --target vendor -r requirements.txt` instead.
## Quick start ## Quick start
Point your browser to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog Point your browser to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog \
Set `Location of Calibre database` to the path of the folder where your Calibre library (metadata.db) lives, push "submit" button\ Login with default admin login \
Optionally a Google Drive can be used to host the calibre library [-> Using Google Drive integration](https://github.com/janeczku/calibre-web/wiki/Configuration#using-google-drive-integration) If you don't have a Calibre database already, this [database](https://github.com/janeczku/calibre-web/blob/master/library/metadata.db) can be used. **IMPORTATNT** Please move the database out of the calibre-web folder structure, as it will be overwritten during update. \
Go to Login page Set `Location of Calibre database` to the path of the folder where your Calibre library (metadata.db) lives, push "submit" button. \
Optionally a Google Drive can be used to host the calibre library [-> Using Google Drive integration](https://github.com/janeczku/calibre-web/wiki/G-Drive-Setup#using-google-drive-integration) \
Afterwards you can configure your Calibre-Web instance ([Basic Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#basic-configuration) and [UI Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#ui-configuration) on admin page)
#### Default admin login: #### Default admin login:
*Username:* admin\ *Username:* admin\
@ -67,11 +66,14 @@ Go to Login page
python 3.5+ python 3.5+
Optionally, to enable on-the-fly conversion from one ebook format to another when using the send-to-kindle feature, or during editing of ebooks metadata: [Download](https://imagemagick.org/script/download.php) Imagemagick to extract covers from epubs. On Windows the additional installation of [ghostscript](https://ghostscript.com/releases/gsdnld.html) might be necessary to extract covers from pdf files. On Linux Imagemagick and Ghostscript can often be installed using the system package manager.
Optionally, to enable on-the-fly conversion from one ebook format to another when using the send-to-ereader feature, or during editing of ebooks metadata:
[Download and install](https://calibre-ebook.com/download) the Calibre desktop program for your platform and enter the folder including program name (normally /opt/calibre/ebook-convert, or C:\Program Files\calibre\ebook-convert.exe) in the field "calibre's converter tool" on the setup page. [Download and install](https://calibre-ebook.com/download) the Calibre desktop program for your platform and enter the folder including program name (normally /opt/calibre/ebook-convert, or C:\Program Files\calibre\ebook-convert.exe) in the field "calibre's converter tool" on the setup page.
[Download](https://github.com/pgaskin/kepubify/releases/latest) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `\opt\kepubify` Windows: `C:\Program Files\kepubify`. [Download](https://github.com/pgaskin/kepubify/releases/latest) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `/opt/kepubify` Windows: `C:\Program Files\kepubify`.
## Docker Images ## Docker Images

46
SECURITY.md
View File

@ -10,20 +10,36 @@ To receive fixes for security vulnerabilities it is required to always upgrade t
## History ## History
| Fixed in | Description |CVE number | | Fixed in | Description |CVE number |
| ---------- |---------|---------| |---------------|--------------------------------------------------------------------------------------------------------------------|---------|
| 3rd July 2018 | Guest access acts as a backdoor|| | 3rd July 2018 | Guest access acts as a backdoor ||
| V 0.6.7 |Hardcoded secret key for sessions |CVE-2020-12627 | | V 0.6.7 | Hardcoded secret key for sessions |CVE-2020-12627 |
| V 0.6.13|Calibre-Web Metadata cross site scripting |CVE-2021-25964| | V 0.6.13 | Calibre-Web Metadata cross site scripting |CVE-2021-25964|
| V 0.6.13|Name of Shelves are only visible to users who can access the corresponding shelf Thanks to @ibarrionuevo|| | V 0.6.13 | Name of Shelves are only visible to users who can access the corresponding shelf Thanks to @ibarrionuevo ||
| V 0.6.13|JavaScript could get executed in the description field. Thanks to @ranjit-git and Hagai Wechsler (WhiteSource)|| | V 0.6.13 | JavaScript could get executed in the description field. Thanks to @ranjit-git and Hagai Wechsler (WhiteSource) ||
| V 0.6.13|JavaScript could get executed in a custom column of type "comment" field || | V 0.6.13 | JavaScript could get executed in a custom column of type "comment" field ||
| V 0.6.13|JavaScript could get executed after converting a book to another format with a title containing javascript code|| | V 0.6.13 | JavaScript could get executed after converting a book to another format with a title containing javascript code ||
| V 0.6.13|JavaScript could get executed after converting a book to another format with a username containing javascript code|| | V 0.6.13 | JavaScript could get executed after converting a book to another format with a username containing javascript code ||
| V 0.6.13|JavaScript could get executed in the description series, categories or publishers title|| | V 0.6.13 | JavaScript could get executed in the description series, categories or publishers title ||
| V 0.6.13|JavaScript could get executed in the shelf title|| | V 0.6.13 | JavaScript could get executed in the shelf title ||
| V 0.6.13|Login with the old session cookie after logout. Thanks to @ibarrionuevo|| | V 0.6.13 | Login with the old session cookie after logout. Thanks to @ibarrionuevo ||
| V 0.6.14|CSRF was possible. Thanks to @mik317 and Hagai Wechsler (WhiteSource) || | V 0.6.14 | CSRF was possible. Thanks to @mik317 and Hagai Wechsler (WhiteSource) |CVE-2021-25965|
| V 0.6.14|Cross-Site Scripting vulnerability on typeahead inputs. Thanks to @notdodo|| | V 0.6.14 | Migrated some routes to POST-requests (CSRF protection). Thanks to @scara31 |CVE-2021-4164|
| V 0.6.15 | Fix for "javascript:" script links in identifier. Thanks to @scara31 |CVE-2021-4170|
| V 0.6.15 | Cross-Site Scripting vulnerability on uploaded cover file names. Thanks to @ibarrionuevo ||
| V 0.6.15 | Creating public shelfs is now denied if user is missing the edit public shelf right. Thanks to @ibarrionuevo ||
| V 0.6.15 | Changed error message in case of trying to delete a shelf unauthorized. Thanks to @ibarrionuevo ||
| V 0.6.16 | JavaScript could get executed on authors page. Thanks to @alicaz |CVE-2022-0352|
| V 0.6.16 | Localhost can no longer be used to upload covers. Thanks to @scara31 |CVE-2022-0339|
| V 0.6.16 | Another case where public shelfs could be created without permission is prevented. Thanks to @nhiephon |CVE-2022-0273|
| V 0.6.16 | It's prevented to get the name of a private shelfs. Thanks to @nhiephon |CVE-2022-0405|
| V 0.6.17 | The SSRF Protection can no longer be bypassed via an HTTP redirect. Thanks to @416e6e61 |CVE-2022-0767|
| V 0.6.17 | The SSRF Protection can no longer be bypassed via 0.0.0.0 and it's ipv6 equivalent. Thanks to @r0hanSH |CVE-2022-0766|
| V 0.6.18 | Possible SQL Injection is prevented in user table Thanks to Iman Sharafaldin (Forward Security) |CVE-2022-30765|
| V 0.6.18 | The SSRF protection no longer can be bypassed by IPV6/IPV4 embedding. Thanks to @416e6e61 |CVE-2022-0939|
| V 0.6.18 | The SSRF protection no longer can be bypassed to connect to other servers in the local network. Thanks to @michaellrowley |CVE-2022-0990|
## Statement regarding Log4j (CVE-2021-44228 and related)
Calibre-web is not affected by bugs related to Log4j. Calibre-Web is a python program, therefore not using Java, and not using the Java logging feature log4j.

3
babel.cfg
View File

@ -1,3 +1,4 @@
[python: **.py] [python: **.py]
# has to be executed with jinja2 >=2.9 to have autoescape enabled automatically
[jinja2: **/templates/**.*ml] [jinja2: **/templates/**.*ml]
extensions=jinja2.ext.autoescape,jinja2.ext.with_

66
cps.py
View File

@ -2,7 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web) # This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2012-2019 OzzieIsaacs # Copyright (C) 2022 OzzieIsaacs
# #
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -17,66 +17,18 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import os import os
import sys
# Insert local directories into path # Add local path to sys.path so we can import cps
sys.path.append(os.path.dirname(os.path.abspath(__file__))) path = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'vendor')) sys.path.insert(0, path)
from cps import create_app
from cps import web_server
from cps.opds import opds
from cps.web import web
from cps.jinjia import jinjia
from cps.about import about
from cps.shelf import shelf
from cps.admin import admi
from cps.gdrive import gdrive
from cps.editbooks import editbook
from cps.remotelogin import remotelogin
from cps.search_metadata import meta
from cps.error_handler import init_errorhandler
try:
from cps.kobo import kobo, get_kobo_activated
from cps.kobo_auth import kobo_auth
kobo_available = get_kobo_activated()
except (ImportError, AttributeError): # Catch also error for not installed flask-WTF (missing csrf decorator)
kobo_available = False
try:
from cps.oauth_bb import oauth
oauth_available = True
except ImportError:
oauth_available = False
def main():
app = create_app()
init_errorhandler()
app.register_blueprint(web)
app.register_blueprint(opds)
app.register_blueprint(jinjia)
app.register_blueprint(about)
app.register_blueprint(shelf)
app.register_blueprint(admi)
app.register_blueprint(remotelogin)
app.register_blueprint(meta)
app.register_blueprint(gdrive)
app.register_blueprint(editbook)
if kobo_available:
app.register_blueprint(kobo)
app.register_blueprint(kobo_auth)
if oauth_available:
app.register_blueprint(oauth)
success = web_server.start()
sys.exit(0 if success else 1)
from cps.main import main
if __name__ == '__main__': if __name__ == '__main__':
main() main()

28
cps/MyLoginManager.py
View File

@ -21,14 +21,32 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
from flask_login import LoginManager from flask_login import LoginManager, confirm_login
from flask import session from flask import session, current_app
from flask_login.utils import decode_cookie
from flask_login.signals import user_loaded_from_cookie
class MyLoginManager(LoginManager): class MyLoginManager(LoginManager):
def _session_protection_failed(self): def _session_protection_failed(self):
sess = session._get_current_object() _session = session._get_current_object()
ident = self._session_identifier_generator() ident = self._session_identifier_generator()
if(sess and not (len(sess) == 1 and sess.get('csrf_token', None))) and ident != sess.get('_id', None): if(_session and not (len(_session) == 1
and _session.get('csrf_token', None))) and ident != _session.get('_id', None):
return super(). _session_protection_failed() return super(). _session_protection_failed()
return False return False
def _load_user_from_remember_cookie(self, cookie):
user_id = decode_cookie(cookie)
if user_id is not None:
session["_user_id"] = user_id
session["_fresh"] = False
user = None
if self._user_callback:
user = self._user_callback(user_id)
if user is not None:
app = current_app._get_current_object()
user_loaded_from_cookie.send(app, user=user)
# if session was restored from remember me cookie make login valid
confirm_login()
return user
return None

143
cps/__init__.py
View File

@ -19,29 +19,27 @@
# #
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
__package__ = "cps"
import sys import sys
import os import os
import mimetypes import mimetypes
from babel import Locale as LC from flask import Flask
from babel import negotiate_locale
from babel.core import UnknownLocaleError
from flask import Flask, request, g
from .MyLoginManager import MyLoginManager from .MyLoginManager import MyLoginManager
from flask_babel import Babel
from flask_principal import Principal from flask_principal import Principal
from . import config_sql, logger, cache_buster, cli, ub, db from . import logger
from .cli import CliParameter
from .constants import CONFIG_DIR
from .reverseproxy import ReverseProxied from .reverseproxy import ReverseProxied
from .server import WebServer from .server import WebServer
from .dep_check import dependency_check from .dep_check import dependency_check
from .updater import Updater
try: from .babel import babel, get_locale
import lxml from . import config_sql
lxml_present = True from . import cache_buster
except ImportError: from . import ub, db
lxml_present = False
try: try:
from flask_wtf.csrf import CSRFProtect from flask_wtf.csrf import CSRFProtect
@ -49,6 +47,7 @@ try:
except ImportError: except ImportError:
wtf_present = False wtf_present = False
mimetypes.init() mimetypes.init()
mimetypes.add_type('application/xhtml+xml', '.xhtml') mimetypes.add_type('application/xhtml+xml', '.xhtml')
mimetypes.add_type('application/epub+zip', '.epub') mimetypes.add_type('application/epub+zip', '.epub')
@ -70,6 +69,8 @@ mimetypes.add_type('application/ogg', '.oga')
mimetypes.add_type('text/css', '.css') mimetypes.add_type('text/css', '.css')
mimetypes.add_type('text/javascript; charset=UTF-8', '.js') mimetypes.add_type('text/javascript; charset=UTF-8', '.js')
log = logger.create()
app = Flask(__name__) app = Flask(__name__)
app.config.update( app.config.update(
SESSION_COOKIE_HTTPONLY=True, SESSION_COOKIE_HTTPONLY=True,
@ -78,61 +79,72 @@ app.config.update(
WTF_CSRF_SSL_STRICT=False WTF_CSRF_SSL_STRICT=False
) )
lm = MyLoginManager() lm = MyLoginManager()
lm.login_view = 'web.login'
lm.anonymous_user = ub.Anonymous config = config_sql._ConfigSQL()
lm.session_protection = 'strong'
cli_param = CliParameter()
if wtf_present: if wtf_present:
csrf = CSRFProtect() csrf = CSRFProtect()
csrf.init_app(app)
else: else:
csrf = None csrf = None
ub.init_db(cli.settingspath) calibre_db = db.CalibreDB()
# pylint: disable=no-member
config = config_sql.load_configuration(ub.session)
web_server = WebServer() web_server = WebServer()
babel = Babel() updater_thread = Updater()
_BABEL_TRANSLATIONS = set()
log = logger.create()
from . import services
db.CalibreDB.update_config(config)
db.CalibreDB.setup_db(config.config_calibre_dir, cli.settingspath)
calibre_db = db.CalibreDB()
def create_app(): def create_app():
lm.login_view = 'web.login'
lm.anonymous_user = ub.Anonymous
lm.session_protection = 'strong'
if csrf:
csrf.init_app(app)
cli_param.init()
ub.init_db(cli_param.settings_path, cli_param.user_credentials)
# pylint: disable=no-member
config_sql.load_configuration(config, ub.session, cli_param)
db.CalibreDB.update_config(config)
db.CalibreDB.setup_db(config.config_calibre_dir, cli_param.settings_path)
calibre_db.init_db()
updater_thread.init_updater(config, web_server)
# Perform dry run of updater and exit afterwards
if cli_param.dry_run:
updater_thread.dry_run()
sys.exit(0)
updater_thread.start()
if sys.version_info < (3, 0): if sys.version_info < (3, 0):
log.info( log.info(
'*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, please update your installation to Python3 ***') '*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, '
'please update your installation to Python3 ***')
print( print(
'*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, please update your installation to Python3 ***') '*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, '
'please update your installation to Python3 ***')
web_server.stop(True) web_server.stop(True)
sys.exit(5) sys.exit(5)
if not lxml_present:
log.info('*** "lxml" is needed for calibre-web to run. Please install it using pip: "pip install lxml" ***')
print('*** "lxml" is needed for calibre-web to run. Please install it using pip: "pip install lxml" ***')
web_server.stop(True)
sys.exit(6)
if not wtf_present: if not wtf_present:
log.info('*** "flask-WTF" is needed for calibre-web to run. Please install it using pip: "pip install flask-WTF" ***') log.info('*** "flask-WTF" is needed for calibre-web to run. '
print('*** "flask-WTF" is needed for calibre-web to run. Please install it using pip: "pip install flask-WTF" ***') 'Please install it using pip: "pip install flask-WTF" ***')
print('*** "flask-WTF" is needed for calibre-web to run. '
'Please install it using pip: "pip install flask-WTF" ***')
web_server.stop(True) web_server.stop(True)
sys.exit(7) sys.exit(7)
for res in dependency_check() + dependency_check(True): for res in dependency_check() + dependency_check(True):
log.info('*** "{}" version does not fit the requirements. Should: {}, Found: {}, please consider installing required version ***' log.info('*** "{}" version does not meet the requirements. '
.format(res['name'], 'Should: {}, Found: {}, please consider installing required version ***'
res['target'], .format(res['name'],
res['found'])) res['target'],
res['found']))
app.wsgi_app = ReverseProxied(app.wsgi_app) app.wsgi_app = ReverseProxied(app.wsgi_app)
if os.environ.get('FLASK_DEBUG'): if os.environ.get('FLASK_DEBUG'):
@ -144,10 +156,13 @@ def create_app():
app.secret_key = os.getenv('SECRET_KEY', config_sql.get_flask_session_key(ub.session)) app.secret_key = os.getenv('SECRET_KEY', config_sql.get_flask_session_key(ub.session))
web_server.init_app(app, config) web_server.init_app(app, config)
if hasattr(babel, "localeselector"):
babel.init_app(app)
babel.localeselector(get_locale)
else:
babel.init_app(app, locale_selector=get_locale)
babel.init_app(app) from . import services
_BABEL_TRANSLATIONS.update(str(item) for item in babel.list_translations())
_BABEL_TRANSLATIONS.add('en')
if services.ldap: if services.ldap:
services.ldap.init_app(app, config) services.ldap.init_app(app, config)
@ -155,34 +170,12 @@ def create_app():
services.goodreads_support.connect(config.config_goodreads_api_key, services.goodreads_support.connect(config.config_goodreads_api_key,
config.config_goodreads_api_secret, config.config_goodreads_api_secret,
config.config_use_goodreads) config.config_use_goodreads)
config.store_calibre_uuid(calibre_db, db.Library_Id)
# Register scheduled tasks
from .schedule import register_scheduled_tasks, register_startup_tasks
register_scheduled_tasks(config.schedule_reconnect)
register_startup_tasks()
return app return app
@babel.localeselector
def get_locale():
# if a user is logged in, use the locale from the user settings
user = getattr(g, 'user', None)
if user is not None and hasattr(user, "locale"):
if user.name != 'Guest': # if the account is the guest account bypass the config lang settings
return user.locale
preferred = list()
if request.accept_languages:
for x in request.accept_languages.values():
try:
preferred.append(str(LC.parse(x.replace('-', '_'))))
except (UnknownLocaleError, ValueError) as e:
log.debug('Could not parse locale "%s": %s', x, e)
return negotiate_locale(preferred or ['en'], _BABEL_TRANSLATIONS)
@babel.timezoneselector
def get_timezone():
user = getattr(g, 'user', None)
return user.timezone if user else None
from .updater import Updater
updater_thread = Updater()
updater_thread.start()

128
cps/about.py
View File

@ -25,114 +25,60 @@ import platform
import sqlite3 import sqlite3
from collections import OrderedDict from collections import OrderedDict
import babel, pytz, requests, sqlalchemy import flask
import werkzeug, flask, flask_login, flask_principal, jinja2 import flask_login
import jinja2
from flask_babel import gettext as _ from flask_babel import gettext as _
try:
from flask_wtf import __version__ as flaskwtf_version
except ImportError:
flaskwtf_version = _(u'not installed')
from . import db, calibre_db, converter, uploader, server, isoLanguages, constants, gdriveutils, dep_check from . import db, calibre_db, converter, uploader, constants, dep_check
from .render_template import render_title_template from .render_template import render_title_template
try:
from flask_login import __version__ as flask_loginVersion
except ImportError:
from flask_login.__about__ import __version__ as flask_loginVersion
try:
# pylint: disable=unused-import
import unidecode
# _() necessary to make babel aware of string for translation
unidecode_version = _(u'installed')
except ImportError:
unidecode_version = _(u'not installed')
try:
from flask_dance import __version__ as flask_danceVersion
except ImportError:
flask_danceVersion = None
try:
from greenlet import __version__ as greenlet_Version
except ImportError:
greenlet_Version = None
try:
from scholarly import scholarly
scholarly_version = _(u'installed')
except ImportError:
scholarly_version = _(u'not installed')
from . import services
about = flask.Blueprint('about', __name__) about = flask.Blueprint('about', __name__)
ret = dict() modules = dict()
req = dep_check.load_dependencys(False) req = dep_check.load_dependencies(False)
opt = dep_check.load_dependencys(True) opt = dep_check.load_dependencies(True)
for i in (req + opt): for i in (req + opt):
ret[i[1]] = i[0] modules[i[1]] = i[0]
modules['Jinja2'] = jinja2.__version__
modules['pySqlite'] = sqlite3.version
modules['SQLite'] = sqlite3.sqlite_version
sorted_modules = OrderedDict((sorted(modules.items(), key=lambda x: x[0].casefold())))
if not ret:
_VERSIONS = OrderedDict(
Platform = '{0[0]} {0[2]} {0[3]} {0[4]} {0[5]}'.format(platform.uname()),
Python=sys.version,
Calibre_Web=constants.STABLE_VERSION['version'] + ' - '
+ constants.NIGHTLY_VERSION[0].replace('%','%%') + ' - '
+ constants.NIGHTLY_VERSION[1].replace('%','%%'),
WebServer=server.VERSION,
Flask=flask.__version__,
Flask_Login=flask_loginVersion,
Flask_Principal=flask_principal.__version__,
Flask_WTF=flaskwtf_version,
Werkzeug=werkzeug.__version__,
Babel=babel.__version__,
Jinja2=jinja2.__version__,
Requests=requests.__version__,
SqlAlchemy=sqlalchemy.__version__,
pySqlite=sqlite3.version,
SQLite=sqlite3.sqlite_version,
iso639=isoLanguages.__version__,
pytz=pytz.__version__,
Unidecode=unidecode_version,
Scholarly=scholarly_version,
Flask_SimpleLDAP=u'installed' if bool(services.ldap) else None,
python_LDAP=services.ldapVersion if bool(services.ldapVersion) else None,
Goodreads=u'installed' if bool(services.goodreads_support) else None,
jsonschema=services.SyncToken.__version__ if bool(services.SyncToken) else None,
flask_dance=flask_danceVersion,
greenlet=greenlet_Version
)
_VERSIONS.update(gdriveutils.get_versions())
_VERSIONS.update(uploader.get_versions(True))
else:
_VERSIONS = OrderedDict(
Platform = '{0[0]} {0[2]} {0[3]} {0[4]} {0[5]}'.format(platform.uname()),
Python = sys.version,
Calibre_Web = constants.STABLE_VERSION['version'] + ' - '
+ constants.NIGHTLY_VERSION[0].replace('%', '%%') + ' - '
+ constants.NIGHTLY_VERSION[1].replace('%', '%%'),
Werkzeug = werkzeug.__version__,
Jinja2=jinja2.__version__,
pySqlite = sqlite3.version,
SQLite = sqlite3.sqlite_version,
)
_VERSIONS.update(ret)
_VERSIONS.update(uploader.get_versions(False))
def collect_stats(): def collect_stats():
_VERSIONS['ebook converter'] = _(converter.get_calibre_version()) if constants.NIGHTLY_VERSION[0] == "$Format:%H$":
_VERSIONS['unrar'] = _(converter.get_unrar_version()) calibre_web_version = constants.STABLE_VERSION['version']
_VERSIONS['kepubify'] = _(converter.get_kepubify_version()) else:
calibre_web_version = (constants.STABLE_VERSION['version'] + ' - '
+ constants.NIGHTLY_VERSION[0].replace('%', '%%') + ' - '
+ constants.NIGHTLY_VERSION[1].replace('%', '%%'))
if getattr(sys, 'frozen', False):
calibre_web_version += " - Exe-Version"
elif constants.HOME_CONFIG:
calibre_web_version += " - pyPi"
_VERSIONS = {'Calibre Web': calibre_web_version}
_VERSIONS.update(OrderedDict(
Python=sys.version,
Platform='{0[0]} {0[2]} {0[3]} {0[4]} {0[5]}'.format(platform.uname()),
))
_VERSIONS.update(uploader.get_magick_version())
_VERSIONS['Unrar'] = converter.get_unrar_version()
_VERSIONS['Ebook converter'] = converter.get_calibre_version()
_VERSIONS['Kepubify'] = converter.get_kepubify_version()
_VERSIONS.update(sorted_modules)
return _VERSIONS return _VERSIONS
@about.route("/stats") @about.route("/stats")
@flask_login.login_required @flask_login.login_required
def stats(): def stats():
counter = calibre_db.session.query(db.Books).count() counter = calibre_db.session.query(db.Books).count()
authors = calibre_db.session.query(db.Authors).count() authors = calibre_db.session.query(db.Authors).count()
categorys = calibre_db.session.query(db.Tags).count() categories = calibre_db.session.query(db.Tags).count()
series = calibre_db.session.query(db.Series).count() series = calibre_db.session.query(db.Series).count()
return render_title_template('stats.html', bookcounter=counter, authorcounter=authors, versions=collect_stats(), return render_title_template('stats.html', bookcounter=counter, authorcounter=authors, versions=collect_stats(),
categorycounter=categorys, seriecounter=series, title=_(u"Statistics"), page="stat") categorycounter=categories, seriecounter=series, title=_("Statistics"), page="stat")

1300
cps/admin.py
View File

File diff suppressed because it is too large Load Diff

40
cps/babel.py Normal file
View File

@ -0,0 +1,40 @@
from babel import negotiate_locale
from flask_babel import Babel, Locale
from babel.core import UnknownLocaleError
from flask import request
from flask_login import current_user
from . import logger
log = logger.create()
babel = Babel()
def get_locale():
# if a user is logged in, use the locale from the user settings
if current_user is not None and hasattr(current_user, "locale"):
# if the account is the guest account bypass the config lang settings
if current_user.name != 'Guest':
return current_user.locale
preferred = list()
if request.accept_languages:
for x in request.accept_languages.values():
try:
preferred.append(str(Locale.parse(x.replace('-', '_'))))
except (UnknownLocaleError, ValueError) as e:
log.debug('Could not parse locale "%s": %s', x, e)
return negotiate_locale(preferred or ['en'], get_available_translations())
def get_user_locale_language(user_language):
return Locale.parse(user_language).get_language_name(get_locale())
def get_available_locale():
return [Locale('en')] + babel.list_translations()
def get_available_translations():
return set(str(item) for item in get_available_locale())

23
cps/cache_buster.py
View File

@ -47,20 +47,23 @@ def init_cache_busting(app):
for filename in filenames: for filename in filenames:
# compute version component # compute version component
rooted_filename = os.path.join(dirpath, filename) rooted_filename = os.path.join(dirpath, filename)
with open(rooted_filename, 'rb') as f: try:
file_hash = hashlib.md5(f.read()).hexdigest()[:7] # nosec with open(rooted_filename, 'rb') as f:
file_hash = hashlib.md5(f.read()).hexdigest()[:7] # nosec
# save version to tables
file_path = rooted_filename.replace(static_folder, "")
file_path = file_path.replace("\\", "/") # Convert Windows path to web path
hash_table[file_path] = file_hash
except PermissionError:
log.error("No permission to access {} file.".format(rooted_filename))
# save version to tables
file_path = rooted_filename.replace(static_folder, "")
file_path = file_path.replace("\\", "/") # Convert Windows path to web path
hash_table[file_path] = file_hash
log.debug('Finished computing cache-busting values') log.debug('Finished computing cache-busting values')
def bust_filename(filename): def bust_filename(file_name):
return hash_table.get(filename, "") return hash_table.get(file_name, "")
def unbust_filename(filename): def unbust_filename(file_name):
return filename.split("?", 1)[0] return file_name.split("?", 1)[0]
@app.url_defaults @app.url_defaults
# pylint: disable=unused-variable # pylint: disable=unused-variable

151
cps/cli.py
View File

@ -24,82 +24,107 @@ import socket
from .constants import CONFIG_DIR as _CONFIG_DIR from .constants import CONFIG_DIR as _CONFIG_DIR
from .constants import STABLE_VERSION as _STABLE_VERSION from .constants import STABLE_VERSION as _STABLE_VERSION
from .constants import NIGHTLY_VERSION as _NIGHTLY_VERSION from .constants import NIGHTLY_VERSION as _NIGHTLY_VERSION
from .constants import DEFAULT_SETTINGS_FILE, DEFAULT_GDRIVE_FILE
def version_info(): def version_info():
if _NIGHTLY_VERSION[1].startswith('$Format'): if _NIGHTLY_VERSION[1].startswith('$Format'):
return "Calibre-Web version: %s - unkown git-clone" % _STABLE_VERSION['version'] return "Calibre-Web version: %s - unknown git-clone" % _STABLE_VERSION['version']
return "Calibre-Web version: %s -%s" % (_STABLE_VERSION['version'], _NIGHTLY_VERSION[1]) return "Calibre-Web version: %s -%s" % (_STABLE_VERSION['version'], _NIGHTLY_VERSION[1])
parser = argparse.ArgumentParser(description='Calibre Web is a web app' class CliParameter(object):
' providing a interface for browsing, reading and downloading eBooks\n', prog='cps.py')
parser.add_argument('-p', metavar='path', help='path and name to settings db, e.g. /opt/cw.db')
parser.add_argument('-g', metavar='path', help='path and name to gdrive db, e.g. /opt/gd.db')
parser.add_argument('-c', metavar='path',
help='path and name to SSL certfile, e.g. /opt/test.cert, works only in combination with keyfile')
parser.add_argument('-k', metavar='path',
help='path and name to SSL keyfile, e.g. /opt/test.key, works only in combination with certfile')
parser.add_argument('-v', '--version', action='version', help='Shows version number and exits Calibre-web',
version=version_info())
parser.add_argument('-i', metavar='ip-address', help='Server IP-Address to listen')
parser.add_argument('-s', metavar='user:pass', help='Sets specific username to new password')
parser.add_argument('-f', action='store_true', help='Flag is depreciated and will be removed in next version')
args = parser.parse_args()
settingspath = args.p or os.path.join(_CONFIG_DIR, "app.db") def init(self):
gdpath = args.g or os.path.join(_CONFIG_DIR, "gdrive.db") self.arg_parser()
# handle and check parameter for ssl encryption def arg_parser(self):
certfilepath = None parser = argparse.ArgumentParser(description='Calibre Web is a web app providing '
keyfilepath = None 'a interface for browsing, reading and downloading eBooks\n',
if args.c: prog='cps.py')
if os.path.isfile(args.c): parser.add_argument('-p', metavar='path', help='path and name to settings db, e.g. /opt/cw.db')
certfilepath = args.c parser.add_argument('-g', metavar='path', help='path and name to gdrive db, e.g. /opt/gd.db')
else: parser.add_argument('-c', metavar='path', help='path and name to SSL certfile, e.g. /opt/test.cert, '
print("Certfile path is invalid. Exiting...") 'works only in combination with keyfile')
sys.exit(1) parser.add_argument('-k', metavar='path', help='path and name to SSL keyfile, e.g. /opt/test.key, '
'works only in combination with certfile')
parser.add_argument('-v', '--version', action='version', help='Shows version number and exits Calibre-Web',
version=version_info())
parser.add_argument('-i', metavar='ip-address', help='Server IP-Address to listen')
parser.add_argument('-s', metavar='user:pass',
help='Sets specific username to new password and exits Calibre-Web')
parser.add_argument('-f', action='store_true', help='Flag is depreciated and will be removed in next version')
parser.add_argument('-l', action='store_true', help='Allow loading covers from localhost')
parser.add_argument('-d', action='store_true', help='Dry run of updater to check file permissions in advance '
'and exits Calibre-Web')
parser.add_argument('-r', action='store_true', help='Enable public database reconnect route under /reconnect')
args = parser.parse_args()
if args.c == "": self.settings_path = args.p or os.path.join(_CONFIG_DIR, DEFAULT_SETTINGS_FILE)
certfilepath = "" self.gd_path = args.g or os.path.join(_CONFIG_DIR, DEFAULT_GDRIVE_FILE)
if args.k: if os.path.isdir(self.settings_path):
if os.path.isfile(args.k): self.settings_path = os.path.join(self.settings_path, DEFAULT_SETTINGS_FILE)
keyfilepath = args.k
else:
print("Keyfile path is invalid. Exiting...")
sys.exit(1)
if (args.k and not args.c) or (not args.k and args.c): if os.path.isdir(self.gd_path):
print("Certfile and Keyfile have to be used together. Exiting...") self.gd_path = os.path.join(self.gd_path, DEFAULT_GDRIVE_FILE)
sys.exit(1)
if args.k == "": # handle and check parameter for ssl encryption
keyfilepath = "" self.certfilepath = None
self.keyfilepath = None
# handle and check ip address argument if args.c:
ip_address = args.i or None if os.path.isfile(args.c):
if ip_address: self.certfilepath = args.c
try:
# try to parse the given ip address with socket
if hasattr(socket, 'inet_pton'):
if ':' in ip_address:
socket.inet_pton(socket.AF_INET6, ip_address)
else: else:
socket.inet_pton(socket.AF_INET, ip_address) print("Certfile path is invalid. Exiting...")
else: sys.exit(1)
# on windows python < 3.4, inet_pton is not available
# inet_atom only handles IPv4 addresses
socket.inet_aton(ip_address)
except socket.error as err:
print(ip_address, ':', err)
sys.exit(1)
# handle and check user password argument if args.c == "":
user_credentials = args.s or None self.certfilepath = ""
if user_credentials and ":" not in user_credentials:
print("No valid 'username:password' format")
sys.exit(3)
if args.f: if args.k:
print("Warning: -f flag is depreciated and will be removed in next version") if os.path.isfile(args.k):
self.keyfilepath = args.k
else:
print("Keyfile path is invalid. Exiting...")
sys.exit(1)
if (args.k and not args.c) or (not args.k and args.c):
print("Certfile and Keyfile have to be used together. Exiting...")
sys.exit(1)
if args.k == "":
self.keyfilepath = ""
# dry run updater
self.dry_run = args.d or None
# enable reconnect endpoint for docker database reconnect
self.reconnect_enable = args.r or os.environ.get("CALIBRE_RECONNECT", None)
# load covers from localhost
self.allow_localhost = args.l or os.environ.get("CALIBRE_LOCALHOST", None)
# handle and check ip address argument
self.ip_address = args.i or None
if self.ip_address:
try:
# try to parse the given ip address with socket
if hasattr(socket, 'inet_pton'):
if ':' in self.ip_address:
socket.inet_pton(socket.AF_INET6, self.ip_address)
else:
socket.inet_pton(socket.AF_INET, self.ip_address)
else:
# on Windows python < 3.4, inet_pton is not available
# inet_atom only handles IPv4 addresses
socket.inet_aton(self.ip_address)
except socket.error as err:
print(self.ip_address, ':', err)
sys.exit(1)
# handle and check user password argument
self.user_credentials = args.s or None
if self.user_credentials and ":" not in self.user_credentials:
print("No valid 'username:password' format")
sys.exit(3)
if args.f:
print("Warning: -f flag is depreciated and will be removed in next version")

94
cps/comic.py
View File

@ -1,7 +1,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web) # This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2018 OzzieIsaacs # Copyright (C) 2018-2022 OzzieIsaacs
# #
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -18,19 +18,16 @@
import os import os
from . import logger, isoLanguages from . import logger, isoLanguages, cover
from .constants import BookMeta from .constants import BookMeta
log = logger.create()
try: try:
from wand.image import Image from wand.image import Image
use_IM = True use_IM = True
except (ImportError, RuntimeError) as e: except (ImportError, RuntimeError) as e:
use_IM = False use_IM = False
log = logger.create()
try: try:
from comicapi.comicarchive import ComicArchive, MetaDataStyle from comicapi.comicarchive import ComicArchive, MetaDataStyle
@ -51,37 +48,16 @@ except (ImportError, LookupError) as e:
use_rarfile = False use_rarfile = False
use_comic_meta = False use_comic_meta = False
NO_JPEG_EXTENSIONS = ['.png', '.webp', '.bmp']
COVER_EXTENSIONS = ['.png', '.webp', '.bmp', '.jpg', '.jpeg']
def _cover_processing(tmp_file_name, img, extension): def _extract_cover_from_archive(original_file_extension, tmp_file_name, rar_executable):
tmp_cover_name = os.path.join(os.path.dirname(tmp_file_name), 'cover.jpg') cover_data = extension = None
if use_IM:
# convert to jpg because calibre only supports jpg
if extension in NO_JPEG_EXTENSIONS:
with Image(filename=tmp_file_name) as imgc:
imgc.format = 'jpeg'
imgc.transform_colorspace('rgb')
imgc.save(tmp_cover_name)
return tmp_cover_name
if not img:
return None
with open(tmp_cover_name, 'wb') as f:
f.write(img)
return tmp_cover_name
def _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecutable):
cover_data = None
if original_file_extension.upper() == '.CBZ': if original_file_extension.upper() == '.CBZ':
cf = zipfile.ZipFile(tmp_file_name) cf = zipfile.ZipFile(tmp_file_name)
for name in cf.namelist(): for name in cf.namelist():
ext = os.path.splitext(name) ext = os.path.splitext(name)
if len(ext) > 1: if len(ext) > 1:
extension = ext[1].lower() extension = ext[1].lower()
if extension in COVER_EXTENSIONS: if extension in cover.COVER_EXTENSIONS:
cover_data = cf.read(name) cover_data = cf.read(name)
break break
elif original_file_extension.upper() == '.CBT': elif original_file_extension.upper() == '.CBT':
@ -90,44 +66,44 @@ def _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecu
ext = os.path.splitext(name) ext = os.path.splitext(name)
if len(ext) > 1: if len(ext) > 1:
extension = ext[1].lower() extension = ext[1].lower()
if extension in COVER_EXTENSIONS: if extension in cover.COVER_EXTENSIONS:
cover_data = cf.extractfile(name).read() cover_data = cf.extractfile(name).read()
break break
elif original_file_extension.upper() == '.CBR' and use_rarfile: elif original_file_extension.upper() == '.CBR' and use_rarfile:
try: try:
rarfile.UNRAR_TOOL = rarExecutable rarfile.UNRAR_TOOL = rar_executable
cf = rarfile.RarFile(tmp_file_name) cf = rarfile.RarFile(tmp_file_name)
for name in cf.getnames(): for name in cf.namelist():
ext = os.path.splitext(name) ext = os.path.splitext(name)
if len(ext) > 1: if len(ext) > 1:
extension = ext[1].lower() extension = ext[1].lower()
if extension in COVER_EXTENSIONS: if extension in cover.COVER_EXTENSIONS:
cover_data = cf.read(name) cover_data = cf.read(name)
break break
except Exception as ex: except Exception as ex:
log.debug('Rarfile failed with error: %s', ex) log.debug('Rarfile failed with error: {}'.format(ex))
return cover_data return cover_data, extension
def _extractCover(tmp_file_name, original_file_extension, rarExecutable): def _extract_cover(tmp_file_name, original_file_extension, rar_executable):
cover_data = extension = None cover_data = extension = None
if use_comic_meta: if use_comic_meta:
archive = ComicArchive(tmp_file_name, rar_exe_path=rarExecutable) archive = ComicArchive(tmp_file_name, rar_exe_path=rar_executable)
for index, name in enumerate(archive.getPageNameList()): for index, name in enumerate(archive.getPageNameList()):
ext = os.path.splitext(name) ext = os.path.splitext(name)
if len(ext) > 1: if len(ext) > 1:
extension = ext[1].lower() extension = ext[1].lower()
if extension in COVER_EXTENSIONS: if extension in cover.COVER_EXTENSIONS:
cover_data = archive.getPage(index) cover_data = archive.getPage(index)
break break
else: else:
cover_data = _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecutable) cover_data, extension = _extract_cover_from_archive(original_file_extension, tmp_file_name, rar_executable)
return _cover_processing(tmp_file_name, cover_data, extension) return cover.cover_processing(tmp_file_name, cover_data, extension)
def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rarExecutable): def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rar_executable):
if use_comic_meta: if use_comic_meta:
archive = ComicArchive(tmp_file_path, rar_exe_path=rarExecutable) archive = ComicArchive(tmp_file_path, rar_exe_path=rar_executable)
if archive.seemsToBeAComicArchive(): if archive.seemsToBeAComicArchive():
if archive.hasMetadata(MetaDataStyle.CIX): if archive.hasMetadata(MetaDataStyle.CIX):
style = MetaDataStyle.CIX style = MetaDataStyle.CIX
@ -137,34 +113,38 @@ def get_comic_info(tmp_file_path, original_file_name, original_file_extension, r
style = None style = None
# if style is not None: # if style is not None:
loadedMetadata = archive.readMetadata(style) loaded_metadata = archive.readMetadata(style)
lang = loadedMetadata.language or "" lang = loaded_metadata.language or ""
loadedMetadata.language = isoLanguages.get_lang3(lang) loaded_metadata.language = isoLanguages.get_lang3(lang)
return BookMeta( return BookMeta(
file_path=tmp_file_path, file_path=tmp_file_path,
extension=original_file_extension, extension=original_file_extension,
title=loadedMetadata.title or original_file_name, title=loaded_metadata.title or original_file_name,
author=" & ".join([credit["person"] author=" & ".join([credit["person"]
for credit in loadedMetadata.credits if credit["role"] == "Writer"]) or u'Unknown', for credit in loaded_metadata.credits if credit["role"] == "Writer"]) or 'Unknown',
cover=_extractCover(tmp_file_path, original_file_extension, rarExecutable), cover=_extract_cover(tmp_file_path, original_file_extension, rar_executable),
description=loadedMetadata.comments or "", description=loaded_metadata.comments or "",
tags="", tags="",
series=loadedMetadata.series or "", series=loaded_metadata.series or "",
series_id=loadedMetadata.issue or "", series_id=loaded_metadata.issue or "",
languages=loadedMetadata.language, languages=loaded_metadata.language,
publisher="") publisher="",
pubdate="",
identifiers=[])
return BookMeta( return BookMeta(
file_path=tmp_file_path, file_path=tmp_file_path,
extension=original_file_extension, extension=original_file_extension,
title=original_file_name, title=original_file_name,
author=u'Unknown', author='Unknown',
cover=_extractCover(tmp_file_path, original_file_extension, rarExecutable), cover=_extract_cover(tmp_file_path, original_file_extension, rar_executable),
description="", description="",
tags="", tags="",
series="", series="",
series_id="", series_id="",
languages="", languages="",
publisher="") publisher="",
pubdate="",
identifiers=[])

104
cps/config_sql.py
View File

@ -29,12 +29,13 @@ try:
except ImportError: except ImportError:
from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.ext.declarative import declarative_base
from . import constants, cli, logger from . import constants, logger
log = logger.create() log = logger.create()
_Base = declarative_base() _Base = declarative_base()
class _Flask_Settings(_Base): class _Flask_Settings(_Base):
__tablename__ = 'flask_settings' __tablename__ = 'flask_settings'
@ -62,18 +63,19 @@ class _Settings(_Base):
mail_gmail_token = Column(JSON, default={}) mail_gmail_token = Column(JSON, default={})
config_calibre_dir = Column(String) config_calibre_dir = Column(String)
config_calibre_uuid = Column(String)
config_port = Column(Integer, default=constants.DEFAULT_PORT) config_port = Column(Integer, default=constants.DEFAULT_PORT)
config_external_port = Column(Integer, default=constants.DEFAULT_PORT) config_external_port = Column(Integer, default=constants.DEFAULT_PORT)
config_certfile = Column(String) config_certfile = Column(String)
config_keyfile = Column(String) config_keyfile = Column(String)
config_trustedhosts = Column(String,default='') config_trustedhosts = Column(String, default='')
config_calibre_web_title = Column(String, default=u'Calibre-Web') config_calibre_web_title = Column(String, default='Calibre-Web')
config_books_per_page = Column(Integer, default=60) config_books_per_page = Column(Integer, default=60)
config_random_books = Column(Integer, default=4) config_random_books = Column(Integer, default=4)
config_authors_max = Column(Integer, default=0) config_authors_max = Column(Integer, default=0)
config_read_column = Column(Integer, default=0) config_read_column = Column(Integer, default=0)
config_title_regex = Column(String, default=r'^(A|The|An|Der|Die|Das|Den|Ein|Eine|Einen|Dem|Des|Einem|Eines)\s+') config_title_regex = Column(String, default=r'^(A|The|An|Der|Die|Das|Den|Ein|Eine|Einen|Dem|Des|Einem|Eines|Le|La|Les|L\'|Un|Une)\s+')
config_mature_content_tags = Column(String, default='') # config_mature_content_tags = Column(String, default='')
config_theme = Column(Integer, default=0) config_theme = Column(Integer, default=0)
config_log_level = Column(SmallInteger, default=logger.DEFAULT_LOG_LEVEL) config_log_level = Column(SmallInteger, default=logger.DEFAULT_LOG_LEVEL)
@ -122,7 +124,7 @@ class _Settings(_Base):
config_ldap_key_path = Column(String, default="") config_ldap_key_path = Column(String, default="")
config_ldap_dn = Column(String, default='dc=example,dc=org') config_ldap_dn = Column(String, default='dc=example,dc=org')
config_ldap_user_object = Column(String, default='uid=%s') config_ldap_user_object = Column(String, default='uid=%s')
config_ldap_member_user_object = Column(String, default='') # config_ldap_member_user_object = Column(String, default='')
config_ldap_openldap = Column(Boolean, default=True) config_ldap_openldap = Column(Boolean, default=True)
config_ldap_group_object_filter = Column(String, default='(&(objectclass=posixGroup)(cn=%s))') config_ldap_group_object_filter = Column(String, default='(&(objectclass=posixGroup)(cn=%s))')
config_ldap_group_members_field = Column(String, default='memberUid') config_ldap_group_members_field = Column(String, default='memberUid')
@ -133,13 +135,19 @@ class _Settings(_Base):
config_calibre = Column(String) config_calibre = Column(String)
config_rarfile_location = Column(String, default=None) config_rarfile_location = Column(String, default=None)
config_upload_formats = Column(String, default=','.join(constants.EXTENSIONS_UPLOAD)) config_upload_formats = Column(String, default=','.join(constants.EXTENSIONS_UPLOAD))
config_unicode_filename =Column(Boolean, default=False) config_unicode_filename = Column(Boolean, default=False)
config_updatechannel = Column(Integer, default=constants.UPDATE_STABLE) config_updatechannel = Column(Integer, default=constants.UPDATE_STABLE)
config_reverse_proxy_login_header_name = Column(String) config_reverse_proxy_login_header_name = Column(String)
config_allow_reverse_proxy_header_login = Column(Boolean, default=False) config_allow_reverse_proxy_header_login = Column(Boolean, default=False)
schedule_start_time = Column(Integer, default=4)
schedule_duration = Column(Integer, default=10)
schedule_generate_book_covers = Column(Boolean, default=False)
schedule_generate_series_covers = Column(Boolean, default=False)
schedule_reconnect = Column(Boolean, default=False)
def __repr__(self): def __repr__(self):
return self.__class__.__name__ return self.__class__.__name__
@ -147,12 +155,16 @@ class _Settings(_Base):
# Class holds all application specific settings in calibre-web # Class holds all application specific settings in calibre-web
class _ConfigSQL(object): class _ConfigSQL(object):
# pylint: disable=no-member # pylint: disable=no-member
def __init__(self, session): def __init__(self):
pass
def init_config(self, session, cli):
self._session = session self._session = session
self._settings = None self._settings = None
self.db_configured = None self.db_configured = None
self.config_calibre_dir = None self.config_calibre_dir = None
self.load() self.load()
self.cli = cli
change = False change = False
if self.config_converterpath == None: # pylint: disable=access-member-before-definition if self.config_converterpath == None: # pylint: disable=access-member-before-definition
@ -160,7 +172,6 @@ class _ConfigSQL(object):
self.config_converterpath = autodetect_calibre_binary() self.config_converterpath = autodetect_calibre_binary()
if self.config_kepubifypath == None: # pylint: disable=access-member-before-definition if self.config_kepubifypath == None: # pylint: disable=access-member-before-definition
change = True change = True
self.config_kepubifypath = autodetect_kepubify_binary() self.config_kepubifypath = autodetect_kepubify_binary()
@ -170,7 +181,6 @@ class _ConfigSQL(object):
if change: if change:
self.save() self.save()
def _read_from_storage(self): def _read_from_storage(self):
if self._settings is None: if self._settings is None:
log.debug("_ConfigSQL._read_from_storage") log.debug("_ConfigSQL._read_from_storage")
@ -178,22 +188,21 @@ class _ConfigSQL(object):
return self._settings return self._settings
def get_config_certfile(self): def get_config_certfile(self):
if cli.certfilepath: if self.cli.certfilepath:
return cli.certfilepath return self.cli.certfilepath
if cli.certfilepath == "": if self.cli.certfilepath == "":
return None return None
return self.config_certfile return self.config_certfile
def get_config_keyfile(self): def get_config_keyfile(self):
if cli.keyfilepath: if self.cli.keyfilepath:
return cli.keyfilepath return self.cli.keyfilepath
if cli.certfilepath == "": if self.cli.certfilepath == "":
return None return None
return self.config_keyfile return self.config_keyfile
@staticmethod def get_config_ipaddress(self):
def get_config_ipaddress(): return self.cli.ip_address or ""
return cli.ip_address or ""
def _has_role(self, role_flag): def _has_role(self, role_flag):
return constants.has_flag(self.config_default_role, role_flag) return constants.has_flag(self.config_default_role, role_flag)
@ -248,12 +257,14 @@ class _ConfigSQL(object):
return logger.get_level_name(self.config_log_level) return logger.get_level_name(self.config_log_level)
def get_mail_settings(self): def get_mail_settings(self):
return {k:v for k, v in self.__dict__.items() if k.startswith('mail_')} return {k: v for k, v in self.__dict__.items() if k.startswith('mail_')}
def get_mail_server_configured(self): def get_mail_server_configured(self):
return bool((self.mail_server != constants.DEFAULT_MAIL_SERVER and self.mail_server_type == 0) return bool((self.mail_server != constants.DEFAULT_MAIL_SERVER and self.mail_server_type == 0)
or (self.mail_gmail_token != {} and self.mail_server_type == 1)) or (self.mail_gmail_token != {} and self.mail_server_type == 1))
def get_scheduled_task_settings(self):
return {k: v for k, v in self.__dict__.items() if k.startswith('schedule_')}
def set_from_dictionary(self, dictionary, field, convertor=None, default=None, encode=None): def set_from_dictionary(self, dictionary, field, convertor=None, default=None, encode=None):
"""Possibly updates a field of this object. """Possibly updates a field of this object.
@ -285,13 +296,12 @@ class _ConfigSQL(object):
def toDict(self): def toDict(self):
storage = {} storage = {}
for k, v in self.__dict__.items(): for k, v in self.__dict__.items():
if k[0] != '_' and not k.endswith("password") and not k.endswith("secret"): if k[0] != '_' and not k.endswith("password") and not k.endswith("secret") and not k == "cli":
storage[k] = v storage[k] = v
return storage return storage
def load(self): def load(self):
'''Load all configuration values from the underlying storage.''' """Load all configuration values from the underlying storage."""
s = self._read_from_storage() # type: _Settings s = self._read_from_storage() # type: _Settings
for k, v in s.__dict__.items(): for k, v in s.__dict__.items():
if k[0] != '_': if k[0] != '_':
@ -304,9 +314,8 @@ class _ConfigSQL(object):
have_metadata_db = bool(self.config_calibre_dir) have_metadata_db = bool(self.config_calibre_dir)
if have_metadata_db: if have_metadata_db:
if not self.config_use_google_drive: db_file = os.path.join(self.config_calibre_dir, 'metadata.db')
db_file = os.path.join(self.config_calibre_dir, 'metadata.db') have_metadata_db = os.path.isfile(db_file)
have_metadata_db = os.path.isfile(db_file)
self.db_configured = have_metadata_db self.db_configured = have_metadata_db
constants.EXTENSIONS_UPLOAD = [x.lstrip().rstrip().lower() for x in self.config_upload_formats.split(',')] constants.EXTENSIONS_UPLOAD = [x.lstrip().rstrip().lower() for x in self.config_upload_formats.split(',')]
if os.environ.get('FLASK_DEBUG'): if os.environ.get('FLASK_DEBUG'):
@ -325,7 +334,7 @@ class _ConfigSQL(object):
self._session.rollback() self._session.rollback()
def save(self): def save(self):
'''Apply all configuration values to the underlying storage.''' """Apply all configuration values to the underlying storage."""
s = self._read_from_storage() # type: _Settings s = self._read_from_storage() # type: _Settings
for k, v in self.__dict__.items(): for k, v in self.__dict__.items():
@ -351,6 +360,15 @@ class _ConfigSQL(object):
# self.config_calibre_dir = None # self.config_calibre_dir = None
self.save() self.save()
def store_calibre_uuid(self, calibre_db, Library_table):
try:
calibre_uuid = calibre_db.session.query(Library_table).one_or_none()
if self.config_calibre_uuid != calibre_uuid.uuid:
self.config_calibre_uuid = calibre_uuid.uuid
self.save()
except AttributeError:
pass
def _migrate_table(session, orm_class): def _migrate_table(session, orm_class):
changed = False changed = False
@ -373,9 +391,9 @@ def _migrate_table(session, orm_class):
else: else:
column_type = column.type column_type = column.type
alter_table = text("ALTER TABLE %s ADD COLUMN `%s` %s %s" % (orm_class.__tablename__, alter_table = text("ALTER TABLE %s ADD COLUMN `%s` %s %s" % (orm_class.__tablename__,
column_name, column_name,
column_type, column_type,
column_default)) column_default))
log.debug(alter_table) log.debug(alter_table)
session.execute(alter_table) session.execute(alter_table)
changed = True changed = True
@ -403,6 +421,7 @@ def autodetect_calibre_binary():
return element return element
return "" return ""
def autodetect_unrar_binary(): def autodetect_unrar_binary():
if sys.platform == "win32": if sys.platform == "win32":
calibre_path = ["C:\\program files\\WinRar\\unRAR.exe", calibre_path = ["C:\\program files\\WinRar\\unRAR.exe",
@ -414,6 +433,7 @@ def autodetect_unrar_binary():
return element return element
return "" return ""
def autodetect_kepubify_binary(): def autodetect_kepubify_binary():
if sys.platform == "win32": if sys.platform == "win32":
calibre_path = ["C:\\program files\\kepubify\\kepubify-windows-64Bit.exe", calibre_path = ["C:\\program files\\kepubify\\kepubify-windows-64Bit.exe",
@ -425,6 +445,7 @@ def autodetect_kepubify_binary():
return element return element
return "" return ""
def _migrate_database(session): def _migrate_database(session):
# make sure the table is created, if it does not exist # make sure the table is created, if it does not exist
_Base.metadata.create_all(session.bind) _Base.metadata.create_all(session.bind)
@ -432,26 +453,21 @@ def _migrate_database(session):
_migrate_table(session, _Flask_Settings) _migrate_table(session, _Flask_Settings)
def load_configuration(session): def load_configuration(conf, session, cli):
_migrate_database(session) _migrate_database(session)
if not session.query(_Settings).count(): if not session.query(_Settings).count():
session.add(_Settings()) session.add(_Settings())
session.commit() session.commit()
conf = _ConfigSQL(session) # conf = _ConfigSQL()
# Migrate from global restrictions to user based restrictions conf.init_config(session, cli)
#if bool(conf.config_default_show & constants.MATURE_CONTENT) and conf.config_denied_tags == "": # return conf
# conf.config_denied_tags = conf.config_mature_content_tags
# conf.save()
# session.query(ub.User).filter(ub.User.mature_content != True). \
# update({"denied_tags": conf.config_mature_content_tags}, synchronize_session=False)
# session.commit()
return conf
def get_flask_session_key(session):
flask_settings = session.query(_Flask_Settings).one_or_none() def get_flask_session_key(_session):
flask_settings = _session.query(_Flask_Settings).one_or_none()
if flask_settings == None: if flask_settings == None:
flask_settings = _Flask_Settings(os.urandom(32)) flask_settings = _Flask_Settings(os.urandom(32))
session.add(flask_settings) _session.add(flask_settings)
session.commit() _session.commit()
return flask_settings.flask_session_key return flask_settings.flask_session_key

49
cps/constants.py
View File

@ -21,28 +21,37 @@ import os
from collections import namedtuple from collections import namedtuple
from sqlalchemy import __version__ as sql_version from sqlalchemy import __version__ as sql_version
sqlalchemy_version2 = ([int(x) for x in sql_version.split('.')] >= [2,0,0]) sqlalchemy_version2 = ([int(x) for x in sql_version.split('.')] >= [2, 0, 0])
# APP_MODE - production, development, or test
APP_MODE = os.environ.get('APP_MODE', 'production')
# if installed via pip this variable is set to true (empty file with name .HOMEDIR present) # if installed via pip this variable is set to true (empty file with name .HOMEDIR present)
HOME_CONFIG = os.path.isfile(os.path.join(os.path.dirname(os.path.abspath(__file__)), '.HOMEDIR')) HOME_CONFIG = os.path.isfile(os.path.join(os.path.dirname(os.path.abspath(__file__)), '.HOMEDIR'))
#In executables updater is not available, so variable is set to False there # In executables updater is not available, so variable is set to False there
UPDATER_AVAILABLE = True UPDATER_AVAILABLE = True
# Base dir is parent of current file, necessary if called from different folder # Base dir is parent of current file, necessary if called from different folder
BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)),os.pardir)) BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir))
STATIC_DIR = os.path.join(BASE_DIR, 'cps', 'static') STATIC_DIR = os.path.join(BASE_DIR, 'cps', 'static')
TEMPLATES_DIR = os.path.join(BASE_DIR, 'cps', 'templates') TEMPLATES_DIR = os.path.join(BASE_DIR, 'cps', 'templates')
TRANSLATIONS_DIR = os.path.join(BASE_DIR, 'cps', 'translations') TRANSLATIONS_DIR = os.path.join(BASE_DIR, 'cps', 'translations')
# Cache dir - use CACHE_DIR environment variable, otherwise use the default directory: cps/cache
DEFAULT_CACHE_DIR = os.path.join(BASE_DIR, 'cps', 'cache')
CACHE_DIR = os.environ.get('CACHE_DIR', DEFAULT_CACHE_DIR)
if HOME_CONFIG: if HOME_CONFIG:
home_dir = os.path.join(os.path.expanduser("~"),".calibre-web") home_dir = os.path.join(os.path.expanduser("~"), ".calibre-web")
if not os.path.exists(home_dir): if not os.path.exists(home_dir):
os.makedirs(home_dir) os.makedirs(home_dir)
CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', home_dir) CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', home_dir)
else: else:
CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', BASE_DIR) CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', BASE_DIR)
DEFAULT_SETTINGS_FILE = "app.db"
DEFAULT_GDRIVE_FILE = "gdrive.db"
ROLE_USER = 0 << 0 ROLE_USER = 0 << 0
ROLE_ADMIN = 1 << 0 ROLE_ADMIN = 1 << 0
@ -133,11 +142,14 @@ except ValueError:
del env_CALIBRE_PORT del env_CALIBRE_PORT
EXTENSIONS_AUDIO = {'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'} EXTENSIONS_AUDIO = {'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'}
EXTENSIONS_CONVERT_FROM = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt','cbz','cbr'] EXTENSIONS_CONVERT_FROM = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf',
EXTENSIONS_CONVERT_TO = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt'] 'txt', 'htmlz', 'rtf', 'odt', 'cbz', 'cbr']
EXTENSIONS_UPLOAD = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'djvu', 'prc', 'doc', 'docx', EXTENSIONS_CONVERT_TO = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2',
'fb2', 'html', 'rtf', 'lit', 'odt', 'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'} 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt']
EXTENSIONS_UPLOAD = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'djvu',
'prc', 'doc', 'docx', 'fb2', 'html', 'rtf', 'lit', 'odt', 'mp3', 'mp4', 'ogg',
'opus', 'wav', 'flac', 'm4a', 'm4b'}
def has_flag(value, bit_flag): def has_flag(value, bit_flag):
@ -149,16 +161,29 @@ def selected_roles(dictionary):
# :rtype: BookMeta # :rtype: BookMeta
BookMeta = namedtuple('BookMeta', 'file_path, extension, title, author, cover, description, tags, series, ' BookMeta = namedtuple('BookMeta', 'file_path, extension, title, author, cover, description, tags, series, '
'series_id, languages, publisher') 'series_id, languages, publisher, pubdate, identifiers')
STABLE_VERSION = {'version': '0.6.15 Beta'} STABLE_VERSION = {'version': '0.6.20 Beta'}
NIGHTLY_VERSION = {} NIGHTLY_VERSION = dict()
NIGHTLY_VERSION[0] = '$Format:%H$' NIGHTLY_VERSION[0] = '$Format:%H$'
NIGHTLY_VERSION[1] = '$Format:%cI$' NIGHTLY_VERSION[1] = '$Format:%cI$'
# NIGHTLY_VERSION[0] = 'bb7d2c6273ae4560e83950d36d64533343623a57' # NIGHTLY_VERSION[0] = 'bb7d2c6273ae4560e83950d36d64533343623a57'
# NIGHTLY_VERSION[1] = '2018-09-09T10:13:08+02:00' # NIGHTLY_VERSION[1] = '2018-09-09T10:13:08+02:00'
# CACHE
CACHE_TYPE_THUMBNAILS = 'thumbnails'
# Thumbnail Types
THUMBNAIL_TYPE_COVER = 1
THUMBNAIL_TYPE_SERIES = 2
THUMBNAIL_TYPE_AUTHOR = 3
# Thumbnails Sizes
COVER_THUMBNAIL_ORIGINAL = 0
COVER_THUMBNAIL_SMALL = 1
COVER_THUMBNAIL_MEDIUM = 2
COVER_THUMBNAIL_LARGE = 3
# clean-up the module namespace # clean-up the module namespace
del sys, os, namedtuple del sys, os, namedtuple

23
cps/converter.py
View File

@ -18,7 +18,8 @@
import os import os
import re import re
from flask_babel import gettext as _
from flask_babel import lazy_gettext as N_
from . import config, logger from . import config, logger
from .subproc_wrapper import process_wait from .subproc_wrapper import process_wait
@ -26,10 +27,9 @@ from .subproc_wrapper import process_wait
log = logger.create() log = logger.create()
# _() necessary to make babel aware of string for translation # strings getting translated when used
_NOT_CONFIGURED = _('not configured') _NOT_INSTALLED = N_('not installed')
_NOT_INSTALLED = _('not installed') _EXECUTION_ERROR = N_('Execution permissions missing')
_EXECUTION_ERROR = _('Execution permissions missing')
def _get_command_version(path, pattern, argument=None): def _get_command_version(path, pattern, argument=None):
@ -48,14 +48,15 @@ def _get_command_version(path, pattern, argument=None):
def get_calibre_version(): def get_calibre_version():
return _get_command_version(config.config_converterpath, r'ebook-convert.*\(calibre', '--version') \ return _get_command_version(config.config_converterpath, r'ebook-convert.*\(calibre', '--version')
or _NOT_CONFIGURED
def get_unrar_version(): def get_unrar_version():
return _get_command_version(config.config_rarfile_location, r'UNRAR.*\d') or _NOT_CONFIGURED unrar_version = _get_command_version(config.config_rarfile_location, r'UNRAR.*\d')
if unrar_version == "not installed":
unrar_version = _get_command_version(config.config_rarfile_location, r'unrar.*\d', '-V')
return unrar_version
def get_kepubify_version(): def get_kepubify_version():
return _get_command_version(config.config_kepubifypath, r'kepubify\s','--version') or _NOT_CONFIGURED return _get_command_version(config.config_kepubifypath, r'kepubify\s', '--version')

48
cps/cover.py Normal file
View File

@ -0,0 +1,48 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
try:
from wand.image import Image
use_IM = True
except (ImportError, RuntimeError) as e:
use_IM = False
NO_JPEG_EXTENSIONS = ['.png', '.webp', '.bmp']
COVER_EXTENSIONS = ['.png', '.webp', '.bmp', '.jpg', '.jpeg']
def cover_processing(tmp_file_name, img, extension):
tmp_cover_name = os.path.join(os.path.dirname(tmp_file_name), 'cover.jpg')
if extension in NO_JPEG_EXTENSIONS:
if use_IM:
with Image(blob=img) as imgc:
imgc.format = 'jpeg'
imgc.transform_colorspace('rgb')
imgc.save(filename=tmp_cover_name)
return tmp_cover_name
else:
return None
if img:
with open(tmp_cover_name, 'wb') as f:
f.write(img)
return tmp_cover_name
else:
return None

548
cps/db.py
View File

@ -17,13 +17,14 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import os import os
import re import re
import ast
import json import json
from datetime import datetime from datetime import datetime
from urllib.parse import quote
import unidecode
from sqlite3 import OperationalError as sqliteOperationalError
from sqlalchemy import create_engine from sqlalchemy import create_engine
from sqlalchemy import Table, Column, ForeignKey, CheckConstraint from sqlalchemy import Table, Column, ForeignKey, CheckConstraint
from sqlalchemy import String, Integer, Boolean, TIMESTAMP, Float from sqlalchemy import String, Integer, Boolean, TIMESTAMP, Float
@ -40,9 +41,8 @@ from sqlalchemy.pool import StaticPool
from sqlalchemy.sql.expression import and_, true, false, text, func, or_ from sqlalchemy.sql.expression import and_, true, false, text, func, or_
from sqlalchemy.ext.associationproxy import association_proxy from sqlalchemy.ext.associationproxy import association_proxy
from flask_login import current_user from flask_login import current_user
from babel import Locale as LC
from babel.core import UnknownLocaleError
from flask_babel import gettext as _ from flask_babel import gettext as _
from flask_babel import get_locale
from flask import flash from flask import flash
from . import logger, ub, isoLanguages from . import logger, ub, isoLanguages
@ -50,11 +50,6 @@ from .pagination import Pagination
from weakref import WeakSet from weakref import WeakSet
try:
import unidecode
use_unidecode = True
except ImportError:
use_unidecode = False
log = logger.create() log = logger.create()
@ -94,6 +89,12 @@ books_publishers_link = Table('books_publishers_link', Base.metadata,
) )
class Library_Id(Base):
__tablename__ = 'library_id'
id = Column(Integer, primary_key=True)
uuid = Column(String, nullable=False)
class Identifiers(Base): class Identifiers(Base):
__tablename__ = 'identifiers' __tablename__ = 'identifiers'
@ -107,85 +108,91 @@ class Identifiers(Base):
self.type = id_type self.type = id_type
self.book = book self.book = book
def formatType(self): def format_type(self):
format_type = self.type.lower() format_type = self.type.lower()
if format_type == 'amazon': if format_type == 'amazon':
return u"Amazon" return "Amazon"
elif format_type.startswith("amazon_"): elif format_type.startswith("amazon_"):
return u"Amazon.{0}".format(format_type[7:]) return "Amazon.{0}".format(format_type[7:])
elif format_type == "isbn": elif format_type == "isbn":
return u"ISBN" return "ISBN"
elif format_type == "doi": elif format_type == "doi":
return u"DOI" return "DOI"
elif format_type == "douban": elif format_type == "douban":
return u"Douban" return "Douban"
elif format_type == "goodreads": elif format_type == "goodreads":
return u"Goodreads" return "Goodreads"
elif format_type == "babelio": elif format_type == "babelio":
return u"Babelio" return "Babelio"
elif format_type == "google": elif format_type == "google":
return u"Google Books" return "Google Books"
elif format_type == "kobo": elif format_type == "kobo":
return u"Kobo" return "Kobo"
elif format_type == "litres": elif format_type == "litres":
return u"ЛитРес" return "ЛитРес"
elif format_type == "issn": elif format_type == "issn":
return u"ISSN" return "ISSN"
elif format_type == "isfdb": elif format_type == "isfdb":
return u"ISFDB" return "ISFDB"
if format_type == "lubimyczytac": if format_type == "lubimyczytac":
return u"Lubimyczytac" return "Lubimyczytac"
if format_type == "databazeknih":
return "Databáze knih"
else: else:
return self.type return self.type
def __repr__(self): def __repr__(self):
format_type = self.type.lower() format_type = self.type.lower()
if format_type == "amazon" or format_type == "asin": if format_type == "amazon" or format_type == "asin":
return u"https://amazon.com/dp/{0}".format(self.val) return "https://amazon.com/dp/{0}".format(self.val)
elif format_type.startswith('amazon_'): elif format_type.startswith('amazon_'):
return u"https://amazon.{0}/dp/{1}".format(format_type[7:], self.val) return "https://amazon.{0}/dp/{1}".format(format_type[7:], self.val)
elif format_type == "isbn": elif format_type == "isbn":
return u"https://www.worldcat.org/isbn/{0}".format(self.val) return "https://www.worldcat.org/isbn/{0}".format(self.val)
elif format_type == "doi": elif format_type == "doi":
return u"https://dx.doi.org/{0}".format(self.val) return "https://dx.doi.org/{0}".format(self.val)
elif format_type == "goodreads": elif format_type == "goodreads":
return u"https://www.goodreads.com/book/show/{0}".format(self.val) return "https://www.goodreads.com/book/show/{0}".format(self.val)
elif format_type == "babelio": elif format_type == "babelio":
return u"https://www.babelio.com/livres/titre/{0}".format(self.val) return "https://www.babelio.com/livres/titre/{0}".format(self.val)
elif format_type == "douban": elif format_type == "douban":
return u"https://book.douban.com/subject/{0}".format(self.val) return "https://book.douban.com/subject/{0}".format(self.val)
elif format_type == "google": elif format_type == "google":
return u"https://books.google.com/books?id={0}".format(self.val) return "https://books.google.com/books?id={0}".format(self.val)
elif format_type == "kobo": elif format_type == "kobo":
return u"https://www.kobo.com/ebook/{0}".format(self.val) return "https://www.kobo.com/ebook/{0}".format(self.val)
elif format_type == "lubimyczytac": elif format_type == "lubimyczytac":
return u"https://lubimyczytac.pl/ksiazka/{0}/ksiazka".format(self.val) return "https://lubimyczytac.pl/ksiazka/{0}/ksiazka".format(self.val)
elif format_type == "litres": elif format_type == "litres":
return u"https://www.litres.ru/{0}".format(self.val) return "https://www.litres.ru/{0}".format(self.val)
elif format_type == "issn": elif format_type == "issn":
return u"https://portal.issn.org/resource/ISSN/{0}".format(self.val) return "https://portal.issn.org/resource/ISSN/{0}".format(self.val)
elif format_type == "isfdb": elif format_type == "isfdb":
return u"http://www.isfdb.org/cgi-bin/pl.cgi?{0}".format(self.val) return "http://www.isfdb.org/cgi-bin/pl.cgi?{0}".format(self.val)
elif format_type == "databazeknih":
return "https://www.databazeknih.cz/knihy/{0}".format(self.val)
elif self.val.lower().startswith("javascript:"):
return quote(self.val)
else: else:
return u"{0}".format(self.val) return "{0}".format(self.val)
class Comments(Base): class Comments(Base):
__tablename__ = 'comments' __tablename__ = 'comments'
id = Column(Integer, primary_key=True) id = Column(Integer, primary_key=True)
book = Column(Integer, ForeignKey('books.id'), nullable=False, unique=True)
text = Column(String(collation='NOCASE'), nullable=False) text = Column(String(collation='NOCASE'), nullable=False)
book = Column(Integer, ForeignKey('books.id'), nullable=False)
def __init__(self, text, book): def __init__(self, comment, book):
self.text = text self.text = comment
self.book = book self.book = book
def get(self): def get(self):
return self.text return self.text
def __repr__(self): def __repr__(self):
return u"<Comments({0})>".format(self.text) return "<Comments({0})>".format(self.text)
class Tags(Base): class Tags(Base):
@ -201,7 +208,7 @@ class Tags(Base):
return self.name return self.name
def __repr__(self): def __repr__(self):
return u"<Tags('{0})>".format(self.name) return "<Tags('{0})>".format(self.name)
class Authors(Base): class Authors(Base):
@ -221,7 +228,7 @@ class Authors(Base):
return self.name return self.name
def __repr__(self): def __repr__(self):
return u"<Authors('{0},{1}{2}')>".format(self.name, self.sort, self.link) return "<Authors('{0},{1}{2}')>".format(self.name, self.sort, self.link)
class Series(Base): class Series(Base):
@ -239,7 +246,7 @@ class Series(Base):
return self.name return self.name
def __repr__(self): def __repr__(self):
return u"<Series('{0},{1}')>".format(self.name, self.sort) return "<Series('{0},{1}')>".format(self.name, self.sort)
class Ratings(Base): class Ratings(Base):
@ -255,7 +262,7 @@ class Ratings(Base):
return self.rating return self.rating
def __repr__(self): def __repr__(self):
return u"<Ratings('{0}')>".format(self.rating) return "<Ratings('{0}')>".format(self.rating)
class Languages(Base): class Languages(Base):
@ -274,7 +281,7 @@ class Languages(Base):
return self.lang_code return self.lang_code
def __repr__(self): def __repr__(self):
return u"<Languages('{0}')>".format(self.lang_code) return "<Languages('{0}')>".format(self.lang_code)
class Publishers(Base): class Publishers(Base):
@ -292,7 +299,7 @@ class Publishers(Base):
return self.name return self.name
def __repr__(self): def __repr__(self):
return u"<Publishers('{0},{1}')>".format(self.name, self.sort) return "<Publishers('{0},{1}')>".format(self.name, self.sort)
class Data(Base): class Data(Base):
@ -316,7 +323,16 @@ class Data(Base):
return self.name return self.name
def __repr__(self): def __repr__(self):
return u"<Data('{0},{1}{2}{3}')>".format(self.book, self.format, self.uncompressed_size, self.name) return "<Data('{0},{1}{2}{3}')>".format(self.book, self.format, self.uncompressed_size, self.name)
class Metadata_Dirtied(Base):
__tablename__ = 'metadata_dirtied'
id = Column(Integer, primary_key=True, autoincrement=True)
book = Column(Integer, ForeignKey('books.id'), nullable=False, unique=True)
def __init__(self, book):
self.book = book
class Books(Base): class Books(Base):
@ -338,15 +354,15 @@ class Books(Base):
isbn = Column(String(collation='NOCASE'), default="") isbn = Column(String(collation='NOCASE'), default="")
flags = Column(Integer, nullable=False, default=1) flags = Column(Integer, nullable=False, default=1)
authors = relationship('Authors', secondary=books_authors_link, backref='books') authors = relationship(Authors, secondary=books_authors_link, backref='books')
tags = relationship('Tags', secondary=books_tags_link, backref='books', order_by="Tags.name") tags = relationship(Tags, secondary=books_tags_link, backref='books', order_by="Tags.name")
comments = relationship('Comments', backref='books') comments = relationship(Comments, backref='books')
data = relationship('Data', backref='books') data = relationship(Data, backref='books')
series = relationship('Series', secondary=books_series_link, backref='books') series = relationship(Series, secondary=books_series_link, backref='books')
ratings = relationship('Ratings', secondary=books_ratings_link, backref='books') ratings = relationship(Ratings, secondary=books_ratings_link, backref='books')
languages = relationship('Languages', secondary=books_languages_link, backref='books') languages = relationship(Languages, secondary=books_languages_link, backref='books')
publishers = relationship('Publishers', secondary=books_publishers_link, backref='books') publishers = relationship(Publishers, secondary=books_publishers_link, backref='books')
identifiers = relationship('Identifiers', backref='books') identifiers = relationship(Identifiers, backref='books')
def __init__(self, title, sort, author_sort, timestamp, pubdate, series_index, last_modified, path, has_cover, def __init__(self, title, sort, author_sort, timestamp, pubdate, series_index, last_modified, path, has_cover,
authors, tags, languages=None): authors, tags, languages=None):
@ -360,18 +376,17 @@ class Books(Base):
self.path = path self.path = path
self.has_cover = (has_cover != None) self.has_cover = (has_cover != None)
def __repr__(self): def __repr__(self):
return u"<Books('{0},{1}{2}{3}{4}{5}{6}{7}{8}')>".format(self.title, self.sort, self.author_sort, return "<Books('{0},{1}{2}{3}{4}{5}{6}{7}{8}')>".format(self.title, self.sort, self.author_sort,
self.timestamp, self.pubdate, self.series_index, self.timestamp, self.pubdate, self.series_index,
self.last_modified, self.path, self.has_cover) self.last_modified, self.path, self.has_cover)
@property @property
def atom_timestamp(self): def atom_timestamp(self):
return (self.timestamp.strftime('%Y-%m-%dT%H:%M:%S+00:00') or '') return self.timestamp.strftime('%Y-%m-%dT%H:%M:%S+00:00') or ''
class Custom_Columns(Base): class CustomColumns(Base):
__tablename__ = 'custom_columns' __tablename__ = 'custom_columns'
id = Column(Integer, primary_key=True) id = Column(Integer, primary_key=True)
@ -385,9 +400,33 @@ class Custom_Columns(Base):
normalized = Column(Boolean) normalized = Column(Boolean)
def get_display_dict(self): def get_display_dict(self):
display_dict = ast.literal_eval(self.display) display_dict = json.loads(self.display)
return display_dict return display_dict
def to_json(self, value, extra, sequence):
content = dict()
content['table'] = "custom_column_" + str(self.id)
content['column'] = "value"
content['datatype'] = self.datatype
content['is_multiple'] = None if not self.is_multiple else self.is_multiple
content['kind'] = "field"
content['name'] = self.name
content['search_terms'] = ['#' + self.label]
content['label'] = self.label
content['colnum'] = self.id
content['display'] = self.get_display_dict()
content['is_custom'] = True
content['is_category'] = self.datatype in ['text', 'rating', 'enumeration', 'series']
content['link_column'] = "value"
content['category_sort'] = "value"
content['is_csp'] = False
content['is_editable'] = self.editable
content['rec_index'] = sequence + 22 # toDo why ??
content['#value#'] = value
content['#extra#'] = extra
content['is_multiple2'] = {}
return json.dumps(content, ensure_ascii=False)
class AlchemyEncoder(json.JSONEncoder): class AlchemyEncoder(json.JSONEncoder):
@ -429,7 +468,7 @@ class AlchemyEncoder(json.JSONEncoder):
return json.JSONEncoder.default(self, o) return json.JSONEncoder.default(self, o)
class CalibreDB(): class CalibreDB:
_init = False _init = False
engine = None engine = None
config = None config = None
@ -438,22 +477,27 @@ class CalibreDB():
# instances alive once they reach the end of their respective scopes # instances alive once they reach the end of their respective scopes
instances = WeakSet() instances = WeakSet()
def __init__(self, expire_on_commit=True): def __init__(self, expire_on_commit=True, init=False):
""" Initialize a new CalibreDB session """ Initialize a new CalibreDB session
""" """
self.session = None self.session = None
if init:
self.init_db(expire_on_commit)
def init_db(self, expire_on_commit=True):
if self._init: if self._init:
self.initSession(expire_on_commit) self.init_session(expire_on_commit)
self.instances.add(self) self.instances.add(self)
def initSession(self, expire_on_commit=True): def init_session(self, expire_on_commit=True):
self.session = self.session_factory() self.session = self.session_factory()
self.session.expire_on_commit = expire_on_commit self.session.expire_on_commit = expire_on_commit
self.update_title_sort(self.config) self.update_title_sort(self.config)
@classmethod @classmethod
def setup_db_cc_classes(self, cc): def setup_db_cc_classes(cls, cc):
cc_ids = [] cc_ids = []
books_custom_column_links = {} books_custom_column_links = {}
for row in cc: for row in cc:
@ -524,25 +568,31 @@ class CalibreDB():
return cc_classes return cc_classes
@classmethod @classmethod
def check_valid_db(cls, config_calibre_dir, app_db_path): def check_valid_db(cls, config_calibre_dir, app_db_path, config_calibre_uuid):
if not config_calibre_dir: if not config_calibre_dir:
return False return False, False
dbpath = os.path.join(config_calibre_dir, "metadata.db") dbpath = os.path.join(config_calibre_dir, "metadata.db")
if not os.path.exists(dbpath): if not os.path.exists(dbpath):
return False return False, False
try: try:
check_engine = create_engine('sqlite://', check_engine = create_engine('sqlite://',
echo=False, echo=False,
isolation_level="SERIALIZABLE", isolation_level="SERIALIZABLE",
connect_args={'check_same_thread': False}, connect_args={'check_same_thread': False},
poolclass=StaticPool) poolclass=StaticPool)
with check_engine.begin() as connection: with check_engine.begin() as connection:
connection.execute(text("attach database '{}' as calibre;".format(dbpath))) connection.execute(text("attach database '{}' as calibre;".format(dbpath)))
connection.execute(text("attach database '{}' as app_settings;".format(app_db_path))) connection.execute(text("attach database '{}' as app_settings;".format(app_db_path)))
local_session = scoped_session(sessionmaker())
local_session.configure(bind=connection)
database_uuid = local_session().query(Library_Id).one_or_none()
# local_session.dispose()
check_engine.connect() check_engine.connect()
db_change = config_calibre_uuid != database_uuid.uuid
except Exception: except Exception:
return False return False, False
return True return True, db_change
@classmethod @classmethod
def update_config(cls, config): def update_config(cls, config):
@ -550,19 +600,16 @@ class CalibreDB():
@classmethod @classmethod
def setup_db(cls, config_calibre_dir, app_db_path): def setup_db(cls, config_calibre_dir, app_db_path):
# cls.config = config
cls.dispose() cls.dispose()
# toDo: if db changed -> delete shelfs, delete download books, delete read boks, kobo sync??
if not config_calibre_dir: if not config_calibre_dir:
cls.config.invalidate() cls.config.invalidate()
return False return None
dbpath = os.path.join(config_calibre_dir, "metadata.db") dbpath = os.path.join(config_calibre_dir, "metadata.db")
if not os.path.exists(dbpath): if not os.path.exists(dbpath):
cls.config.invalidate() cls.config.invalidate()
return False return None
try: try:
cls.engine = create_engine('sqlite://', cls.engine = create_engine('sqlite://',
@ -578,7 +625,7 @@ class CalibreDB():
# conn.text_factory = lambda b: b.decode(errors = 'ignore') possible fix for #1302 # conn.text_factory = lambda b: b.decode(errors = 'ignore') possible fix for #1302
except Exception as ex: except Exception as ex:
cls.config.invalidate(ex) cls.config.invalidate(ex)
return False return None
cls.config.db_configured = True cls.config.db_configured = True
@ -587,16 +634,16 @@ class CalibreDB():
cc = conn.execute(text("SELECT id, datatype FROM custom_columns")) cc = conn.execute(text("SELECT id, datatype FROM custom_columns"))
cls.setup_db_cc_classes(cc) cls.setup_db_cc_classes(cc)
except OperationalError as e: except OperationalError as e:
log.debug_or_exception(e) log.error_or_exception(e)
return None
cls.session_factory = scoped_session(sessionmaker(autocommit=False, cls.session_factory = scoped_session(sessionmaker(autocommit=False,
autoflush=True, autoflush=True,
bind=cls.engine)) bind=cls.engine))
for inst in cls.instances: for inst in cls.instances:
inst.initSession() inst.init_session()
cls._init = True cls._init = True
return True
def get_book(self, book_id): def get_book(self, book_id):
return self.session.query(Books).filter(Books.id == book_id).first() return self.session.query(Books).filter(Books.id == book_id).first()
@ -605,30 +652,60 @@ class CalibreDB():
return self.session.query(Books).filter(Books.id == book_id). \ return self.session.query(Books).filter(Books.id == book_id). \
filter(self.common_filters(allow_show_archived)).first() filter(self.common_filters(allow_show_archived)).first()
def get_book_read_archived(self, book_id, read_column, allow_show_archived=False):
if not read_column:
bd = (self.session.query(Books, ub.ReadBook.read_status, ub.ArchivedBook.is_archived).select_from(Books)
.join(ub.ReadBook, and_(ub.ReadBook.user_id == int(current_user.id), ub.ReadBook.book_id == book_id),
isouter=True))
else:
try:
read_column = cc_classes[read_column]
bd = (self.session.query(Books, read_column.value, ub.ArchivedBook.is_archived).select_from(Books)
.join(read_column, read_column.book == book_id,
isouter=True))
except (KeyError, AttributeError, IndexError):
log.error("Custom Column No.{} does not exist in calibre database".format(read_column))
# Skip linking read column and return None instead of read status
bd = self.session.query(Books, None, ub.ArchivedBook.is_archived)
return (bd.filter(Books.id == book_id)
.join(ub.ArchivedBook, and_(Books.id == ub.ArchivedBook.book_id,
int(current_user.id) == ub.ArchivedBook.user_id), isouter=True)
.filter(self.common_filters(allow_show_archived)).first())
def get_book_by_uuid(self, book_uuid): def get_book_by_uuid(self, book_uuid):
return self.session.query(Books).filter(Books.uuid == book_uuid).first() return self.session.query(Books).filter(Books.uuid == book_uuid).first()
def get_book_format(self, book_id, file_format): def get_book_format(self, book_id, file_format):
return self.session.query(Data).filter(Data.book == book_id).filter(Data.format == file_format).first() return self.session.query(Data).filter(Data.book == book_id).filter(Data.format == file_format).first()
def set_metadata_dirty(self, book_id):
if not self.session.query(Metadata_Dirtied).filter(Metadata_Dirtied.book == book_id).one_or_none():
self.session.add(Metadata_Dirtied(book_id))
def delete_dirty_metadata(self, book_id):
try:
self.session.query(Metadata_Dirtied).filter(Metadata_Dirtied.book == book_id).delete()
self.session.commit()
except (OperationalError) as e:
self.session.rollback()
log.error("Database error: {}".format(e))
# Language and content filters for displaying in the UI # Language and content filters for displaying in the UI
def common_filters(self, allow_show_archived=False): def common_filters(self, allow_show_archived=False, return_all_languages=False):
if not allow_show_archived: if not allow_show_archived:
archived_books = ( archived_books = (ub.session.query(ub.ArchivedBook)
ub.session.query(ub.ArchivedBook) .filter(ub.ArchivedBook.user_id == int(current_user.id))
.filter(ub.ArchivedBook.user_id == int(current_user.id)) .filter(ub.ArchivedBook.is_archived == True)
.filter(ub.ArchivedBook.is_archived == True) .all())
.all()
)
archived_book_ids = [archived_book.book_id for archived_book in archived_books] archived_book_ids = [archived_book.book_id for archived_book in archived_books]
archived_filter = Books.id.notin_(archived_book_ids) archived_filter = Books.id.notin_(archived_book_ids)
else: else:
archived_filter = true() archived_filter = true()
if current_user.filter_language() != "all": if current_user.filter_language() == "all" or return_all_languages:
lang_filter = Books.languages.any(Languages.lang_code == current_user.filter_language())
else:
lang_filter = true() lang_filter = true()
else:
lang_filter = Books.languages.any(Languages.lang_code == current_user.filter_language())
negtags_list = current_user.list_denied_tags() negtags_list = current_user.list_denied_tags()
postags_list = current_user.list_allowed_tags() postags_list = current_user.list_allowed_tags()
neg_content_tags_filter = false() if negtags_list == [''] else Books.tags.any(Tags.name.in_(negtags_list)) neg_content_tags_filter = false() if negtags_list == [''] else Books.tags.any(Tags.name.in_(negtags_list))
@ -638,17 +715,17 @@ class CalibreDB():
pos_cc_list = current_user.allowed_column_value.split(',') pos_cc_list = current_user.allowed_column_value.split(',')
pos_content_cc_filter = true() if pos_cc_list == [''] else \ pos_content_cc_filter = true() if pos_cc_list == [''] else \
getattr(Books, 'custom_column_' + str(self.config.config_restricted_column)). \ getattr(Books, 'custom_column_' + str(self.config.config_restricted_column)). \
any(cc_classes[self.config.config_restricted_column].value.in_(pos_cc_list)) any(cc_classes[self.config.config_restricted_column].value.in_(pos_cc_list))
neg_cc_list = current_user.denied_column_value.split(',') neg_cc_list = current_user.denied_column_value.split(',')
neg_content_cc_filter = false() if neg_cc_list == [''] else \ neg_content_cc_filter = false() if neg_cc_list == [''] else \
getattr(Books, 'custom_column_' + str(self.config.config_restricted_column)). \ getattr(Books, 'custom_column_' + str(self.config.config_restricted_column)). \
any(cc_classes[self.config.config_restricted_column].value.in_(neg_cc_list)) any(cc_classes[self.config.config_restricted_column].value.in_(neg_cc_list))
except (KeyError, AttributeError): except (KeyError, AttributeError, IndexError):
pos_content_cc_filter = false() pos_content_cc_filter = false()
neg_content_cc_filter = true() neg_content_cc_filter = true()
log.error(u"Custom Column No.%d is not existing in calibre database", log.error("Custom Column No.{} does not exist in calibre database".format(
self.config.config_restricted_column) self.config.config_restricted_column))
flash(_("Custom Column No.%(column)d is not existing in calibre database", flash(_("Custom Column No.%(column)d does not exist in calibre database",
column=self.config.config_restricted_column), column=self.config.config_restricted_column),
category="error") category="error")
@ -658,10 +735,32 @@ class CalibreDB():
return and_(lang_filter, pos_content_tags_filter, ~neg_content_tags_filter, return and_(lang_filter, pos_content_tags_filter, ~neg_content_tags_filter,
pos_content_cc_filter, ~neg_content_cc_filter, archived_filter) pos_content_cc_filter, ~neg_content_cc_filter, archived_filter)
def generate_linked_query(self, config_read_column, database):
if not config_read_column:
query = (self.session.query(database, ub.ArchivedBook.is_archived, ub.ReadBook.read_status)
.select_from(Books)
.outerjoin(ub.ReadBook,
and_(ub.ReadBook.user_id == int(current_user.id), ub.ReadBook.book_id == Books.id)))
else:
try:
read_column = cc_classes[config_read_column]
query = (self.session.query(database, ub.ArchivedBook.is_archived, read_column.value)
.select_from(Books)
.outerjoin(read_column, read_column.book == Books.id))
except (KeyError, AttributeError, IndexError):
log.error("Custom Column No.{} does not exist in calibre database".format(config_read_column))
# Skip linking read column and return None instead of read status
query = self.session.query(database, None, ub.ArchivedBook.is_archived)
return query.outerjoin(ub.ArchivedBook, and_(Books.id == ub.ArchivedBook.book_id,
int(current_user.id) == ub.ArchivedBook.user_id))
@staticmethod @staticmethod
def get_checkbox_sorted(inputlist, state, offset, limit, order): def get_checkbox_sorted(inputlist, state, offset, limit, order, combo=False):
outcome = list() outcome = list()
elementlist = {ele.id: ele for ele in inputlist} if combo:
elementlist = {ele[0].id: ele for ele in inputlist}
else:
elementlist = {ele.id: ele for ele in inputlist}
for entry in state: for entry in state:
try: try:
outcome.append(elementlist[entry]) outcome.append(elementlist[entry])
@ -675,33 +774,42 @@ class CalibreDB():
return outcome[offset:offset + limit] return outcome[offset:offset + limit]
# Fill indexpage with all requested data from database # Fill indexpage with all requested data from database
def fill_indexpage(self, page, pagesize, database, db_filter, order, *join): def fill_indexpage(self, page, pagesize, database, db_filter, order,
return self.fill_indexpage_with_archived_books(page, pagesize, database, db_filter, order, False, *join) join_archive_read=False, config_read_column=0, *join):
return self.fill_indexpage_with_archived_books(page, database, pagesize, db_filter, order, False,
join_archive_read, config_read_column, *join)
def fill_indexpage_with_archived_books(self, page, pagesize, database, db_filter, order, allow_show_archived, def fill_indexpage_with_archived_books(self, page, database, pagesize, db_filter, order, allow_show_archived,
*join): join_archive_read, config_read_column, *join):
pagesize = pagesize or self.config.config_books_per_page pagesize = pagesize or self.config.config_books_per_page
if current_user.show_detail_random(): if current_user.show_detail_random():
randm = self.session.query(Books) \ random_query = self.generate_linked_query(config_read_column, database)
.filter(self.common_filters(allow_show_archived)) \ randm = (random_query.filter(self.common_filters(allow_show_archived))
.order_by(func.random()) \ .order_by(func.random())
.limit(self.config.config_random_books).all() .limit(self.config.config_random_books).all())
else: else:
randm = false() randm = false()
if join_archive_read:
query = self.generate_linked_query(config_read_column, database)
else:
query = self.session.query(database)
off = int(int(pagesize) * (page - 1)) off = int(int(pagesize) * (page - 1))
query = self.session.query(database)
if len(join) == 6: indx = len(join)
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4]).outerjoin(join[5]) element = 0
if len(join) == 5: while indx:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4]) if indx >= 3:
if len(join) == 4: query = query.outerjoin(join[element], join[element+1]).outerjoin(join[element+2])
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3]) indx -= 3
if len(join) == 3: element += 3
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]) elif indx == 2:
elif len(join) == 2: query = query.outerjoin(join[element], join[element+1])
query = query.outerjoin(join[0], join[1]) indx -= 2
elif len(join) == 1: element += 2
query = query.outerjoin(join[0]) elif indx == 1:
query = query.outerjoin(join[element])
indx -= 1
element += 1
query = query.filter(db_filter)\ query = query.filter(db_filter)\
.filter(self.common_filters(allow_show_archived)) .filter(self.common_filters(allow_show_archived))
entries = list() entries = list()
@ -711,29 +819,48 @@ class CalibreDB():
len(query.all())) len(query.all()))
entries = query.order_by(*order).offset(off).limit(pagesize).all() entries = query.order_by(*order).offset(off).limit(pagesize).all()
except Exception as ex: except Exception as ex:
log.debug_or_exception(ex) log.error_or_exception(ex)
#for book in entries: # display authors in right order
# book = self.order_authors(book) entries = self.order_authors(entries, True, join_archive_read)
return entries, randm, pagination return entries, randm, pagination
# Orders all Authors in the list according to authors sort # Orders all Authors in the list according to authors sort
def order_authors(self, entry): def order_authors(self, entries, list_return=False, combined=False):
sort_authors = entry.author_sort.split('&') # entries_copy = copy.deepcopy(entries)
authors_ordered = list() # entries_copy =[]
error = False for entry in entries:
ids = [a.id for a in entry.authors] if combined:
for auth in sort_authors: sort_authors = entry.Books.author_sort.split('&')
results = self.session.query(Authors).filter(Authors.sort == auth.lstrip().strip()).all() ids = [a.id for a in entry.Books.authors]
# ToDo: How to handle not found authorname
if not len(results): else:
error = True sort_authors = entry.author_sort.split('&')
break ids = [a.id for a in entry.authors]
for r in results: authors_ordered = list()
if r.id in ids: # error = False
authors_ordered.append(r) for auth in sort_authors:
if not error: results = self.session.query(Authors).filter(Authors.sort == auth.lstrip().strip()).all()
entry.authors = authors_ordered # ToDo: How to handle not found author name
return entry if not len(results):
log.error("Author {} not found to display name in right order".format(auth.strip()))
# error = True
break
for r in results:
if r.id in ids:
authors_ordered.append(r)
ids.remove(r.id)
for author_id in ids:
result = self.session.query(Authors).filter(Authors.id == author_id).first()
authors_ordered.append(result)
if list_return:
if combined:
entry.Books.authors = authors_ordered
else:
entry.ordered_authors = authors_ordered
else:
return authors_ordered
return entries
def get_typeahead(self, database, query, replace=('', ''), tag_filter=true()): def get_typeahead(self, database, query, replace=('', ''), tag_filter=true()):
query = query or '' query = query or ''
@ -747,21 +874,21 @@ class CalibreDB():
def check_exists_book(self, authr, title): def check_exists_book(self, authr, title):
self.session.connection().connection.connection.create_function("lower", 1, lcase) self.session.connection().connection.connection.create_function("lower", 1, lcase)
q = list() q = list()
authorterms = re.split(r'\s*&\s*', authr) author_terms = re.split(r'\s*&\s*', authr)
for authorterm in authorterms: for author_term in author_terms:
q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + authorterm + "%"))) q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + author_term + "%")))
return self.session.query(Books) \ return self.session.query(Books) \
.filter(and_(Books.authors.any(and_(*q)), func.lower(Books.title).ilike("%" + title + "%"))).first() .filter(and_(Books.authors.any(and_(*q)), func.lower(Books.title).ilike("%" + title + "%"))).first()
def search_query(self, term, *join): def search_query(self, term, config, *join):
term.strip().lower() term.strip().lower()
self.session.connection().connection.connection.create_function("lower", 1, lcase) self.session.connection().connection.connection.create_function("lower", 1, lcase)
q = list() q = list()
authorterms = re.split("[, ]+", term) author_terms = re.split("[, ]+", term)
for authorterm in authorterms: for author_term in author_terms:
q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + authorterm + "%"))) q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + author_term + "%")))
query = self.session.query(Books) query = self.generate_linked_query(config.config_read_column, Books)
if len(join) == 6: if len(join) == 6:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4]).outerjoin(join[5]) query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4]).outerjoin(join[5])
if len(join) == 3: if len(join) == 3:
@ -770,19 +897,42 @@ class CalibreDB():
query = query.outerjoin(join[0], join[1]) query = query.outerjoin(join[0], join[1])
elif len(join) == 1: elif len(join) == 1:
query = query.outerjoin(join[0]) query = query.outerjoin(join[0])
return query.filter(self.common_filters(True)).filter(
or_(Books.tags.any(func.lower(Tags.name).ilike("%" + term + "%")), cc = self.get_cc_columns(config, filter_config_custom_read=True)
Books.series.any(func.lower(Series.name).ilike("%" + term + "%")), filter_expression = [Books.tags.any(func.lower(Tags.name).ilike("%" + term + "%")),
Books.authors.any(and_(*q)), Books.series.any(func.lower(Series.name).ilike("%" + term + "%")),
Books.publishers.any(func.lower(Publishers.name).ilike("%" + term + "%")), Books.authors.any(and_(*q)),
func.lower(Books.title).ilike("%" + term + "%") Books.publishers.any(func.lower(Publishers.name).ilike("%" + term + "%")),
)) func.lower(Books.title).ilike("%" + term + "%")]
for c in cc:
if c.datatype not in ["datetime", "rating", "bool", "int", "float"]:
filter_expression.append(
getattr(Books,
'custom_column_' + str(c.id)).any(
func.lower(cc_classes[c.id].value).ilike("%" + term + "%")))
return query.filter(self.common_filters(True)).filter(or_(*filter_expression))
def get_cc_columns(self, config, filter_config_custom_read=False):
tmp_cc = self.session.query(CustomColumns).filter(CustomColumns.datatype.notin_(cc_exceptions)).all()
cc = []
r = None
if config.config_columns_to_ignore:
r = re.compile(config.config_columns_to_ignore)
for col in tmp_cc:
if filter_config_custom_read and config.config_read_column and config.config_read_column == col.id:
continue
if r and r.match(col.name):
continue
cc.append(col)
return cc
# read search results from calibre-database and return it (function is used for feed and simple search # read search results from calibre-database and return it (function is used for feed and simple search
def get_search_results(self, term, offset=None, order=None, limit=None, *join): def get_search_results(self, term, config, offset=None, order=None, limit=None, *join):
order = order[0] if order else [Books.sort] order = order[0] if order else [Books.sort]
pagination = None pagination = None
result = self.search_query(term, *join).order_by(*order).all() result = self.search_query(term, config, *join).order_by(*order).all()
result_count = len(result) result_count = len(result)
if offset != None and limit != None: if offset != None and limit != None:
offset = int(offset) offset = int(offset)
@ -792,22 +942,44 @@ class CalibreDB():
offset = 0 offset = 0
limit_all = result_count limit_all = result_count
ub.store_ids(result) ub.store_combo_ids(result)
return result[offset:limit_all], result_count, pagination entries = self.order_authors(result[offset:limit_all], list_return=True, combined=True)
return entries, result_count, pagination
# Creates for all stored languages a translated speaking name in the array for the UI # Creates for all stored languages a translated speaking name in the array for the UI
def speaking_language(self, languages=None): def speaking_language(self, languages=None, return_all_languages=False, with_count=False, reverse_order=False):
from . import get_locale
if not languages: if with_count:
languages = self.session.query(Languages) \ if not languages:
.join(books_languages_link) \ languages = self.session.query(Languages, func.count('books_languages_link.book'))\
.join(Books) \ .join(books_languages_link).join(Books)\
.filter(self.common_filters()) \ .filter(self.common_filters(return_all_languages=return_all_languages)) \
.group_by(text('books_languages_link.lang_code')).all() .group_by(text('books_languages_link.lang_code')).all()
for lang in languages: tags = list()
lang.name = isoLanguages.get_language_name(get_locale(), lang.lang_code) for lang in languages:
return languages tag = Category(isoLanguages.get_language_name(get_locale(), lang[0].lang_code), lang[0].lang_code)
tags.append([tag, lang[1]])
# Append all books without language to list
if not return_all_languages:
no_lang_count = (self.session.query(Books)
.outerjoin(books_languages_link).outerjoin(Languages)
.filter(Languages.lang_code == None)
.filter(self.common_filters())
.count())
if no_lang_count:
tags.append([Category(_("None"), "none"), no_lang_count])
return sorted(tags, key=lambda x: x[0].name.lower(), reverse=reverse_order)
else:
if not languages:
languages = self.session.query(Languages) \
.join(books_languages_link) \
.join(Books) \
.filter(self.common_filters(return_all_languages=return_all_languages)) \
.group_by(text('books_languages_link.lang_code')).all()
for lang in languages:
lang.name = isoLanguages.get_language_name(get_locale(), lang.lang_code)
return sorted(languages, key=lambda x: x.name, reverse=reverse_order)
def update_title_sort(self, config, conn=None): def update_title_sort(self, config, conn=None):
# user defined sort function for calibre databases (Series, etc.) # user defined sort function for calibre databases (Series, etc.)
@ -821,7 +993,10 @@ class CalibreDB():
return title.strip() return title.strip()
conn = conn or self.session.connection().connection.connection conn = conn or self.session.connection().connection.connection
conn.create_function("title_sort", 1, _title_sort) try:
conn.create_function("title_sort", 1, _title_sort)
except sqliteOperationalError:
pass
@classmethod @classmethod
def dispose(cls): def dispose(cls):
@ -866,6 +1041,25 @@ def lcase(s):
try: try:
return unidecode.unidecode(s.lower()) return unidecode.unidecode(s.lower())
except Exception as ex: except Exception as ex:
log = logger.create() _log = logger.create()
log.debug_or_exception(ex) _log.error_or_exception(ex)
return s.lower() return s.lower()
class Category:
name = None
id = None
count = None
rating = None
def __init__(self, name, cat_id, rating=None):
self.name = name
self.id = cat_id
self.rating = rating
self.count = 1
'''class Count:
count = None
def __init__(self, count):
self.count = count'''

12
cps/debug_info.py
View File

@ -22,6 +22,7 @@ import glob
import zipfile import zipfile
import json import json
from io import BytesIO from io import BytesIO
from flask_babel.speaklater import LazyString
import os import os
@ -32,6 +33,13 @@ from .about import collect_stats
log = logger.create() log = logger.create()
class lazyEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, LazyString):
return str(obj)
# Let the base class default method raise the TypeError
return json.JSONEncoder.default(self, obj)
def assemble_logfiles(file_name): def assemble_logfiles(file_name):
log_list = sorted(glob.glob(file_name + '*'), reverse=True) log_list = sorted(glob.glob(file_name + '*'), reverse=True)
wfd = BytesIO() wfd = BytesIO()
@ -57,8 +65,8 @@ def send_debug():
file_list.remove(element) file_list.remove(element)
memory_zip = BytesIO() memory_zip = BytesIO()
with zipfile.ZipFile(memory_zip, 'w', compression=zipfile.ZIP_DEFLATED) as zf: with zipfile.ZipFile(memory_zip, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
zf.writestr('settings.txt', json.dumps(config.toDict())) zf.writestr('settings.txt', json.dumps(config.toDict(), sort_keys=True, indent=2))
zf.writestr('libs.txt', json.dumps(collect_stats())) zf.writestr('libs.txt', json.dumps(collect_stats(), sort_keys=True, indent=2, cls=lazyEncoder))
for fp in file_list: for fp in file_list:
zf.write(fp, os.path.basename(fp)) zf.write(fp, os.path.basename(fp))
memory_zip.seek(0) memory_zip.seek(0)

49
cps/dep_check.py
View File

@ -1,14 +1,16 @@
import os import os
import re import re
import sys
import json
from .constants import BASE_DIR from .constants import BASE_DIR
try: try:
from importlib_metadata import version from importlib.metadata import version
importlib = True importlib = True
ImportNotFound = BaseException ImportNotFound = BaseException
except ImportError: except ImportError:
importlib = False importlib = False
version = None
if not importlib: if not importlib:
try: try:
@ -18,8 +20,16 @@ if not importlib:
except ImportError as e: except ImportError as e:
pkgresources = False pkgresources = False
def load_dependencys(optional=False):
def load_dependencies(optional=False):
deps = list() deps = list()
if getattr(sys, 'frozen', False):
pip_installed = os.path.join(BASE_DIR, ".pip_installed")
if os.path.exists(pip_installed):
with open(pip_installed) as f:
exe_deps = json.loads("".join(f.readlines()))
else:
return deps
if importlib or pkgresources: if importlib or pkgresources:
if optional: if optional:
req_path = os.path.join(BASE_DIR, "optional-requirements.txt") req_path = os.path.join(BASE_DIR, "optional-requirements.txt")
@ -31,11 +41,14 @@ def load_dependencys(optional=False):
if not line.startswith('#') and not line == '\n' and not line.startswith('git'): if not line.startswith('#') and not line == '\n' and not line.startswith('git'):
res = re.match(r'(.*?)([<=>\s]+)([\d\.]+),?\s?([<=>\s]+)?([\d\.]+)?', line.strip()) res = re.match(r'(.*?)([<=>\s]+)([\d\.]+),?\s?([<=>\s]+)?([\d\.]+)?', line.strip())
try: try:
if importlib: if getattr(sys, 'frozen', False):
dep_version = version(res.group(1)) dep_version = exe_deps[res.group(1).lower().replace('_', '-')]
else: else:
dep_version = pkg_resources.get_distribution(res.group(1)).version if importlib:
except ImportNotFound: dep_version = version(res.group(1))
else:
dep_version = pkg_resources.get_distribution(res.group(1)).version
except (ImportNotFound, KeyError):
if optional: if optional:
continue continue
dep_version = "not installed" dep_version = "not installed"
@ -45,38 +58,38 @@ def load_dependencys(optional=False):
def dependency_check(optional=False): def dependency_check(optional=False):
d = list() d = list()
deps = load_dependencys(optional) deps = load_dependencies(optional)
for dep in deps: for dep in deps:
try: try:
dep_version_int = [int(x) for x in dep[0].split('.')] dep_version_int = [int(x) for x in dep[0].split('.')]
low_check = [int(x) for x in dep[3].split('.')] low_check = [int(x) for x in dep[3].split('.')]
high_check = [int(x) for x in dep[5].split('.')] high_check = [int(x) for x in dep[5].split('.')]
except AttributeError: except AttributeError:
high_check = None high_check = []
except ValueError: except ValueError:
d.append({'name': dep[1], d.append({'name': dep[1],
'target': "available", 'target': "available",
'found': "Not available" 'found': "Not available"
}) })
continue continue
if dep[2].strip() == "==": if dep[2].strip() == "==":
if dep_version_int != low_check: if dep_version_int != low_check:
d.append({'name': dep[1], d.append({'name': dep[1],
'found': dep[0], 'found': dep[0],
"target": dep[2] + dep[3]}) "target": dep[2] + dep[3]})
continue continue
elif dep[2].strip() == ">=": elif dep[2].strip() == ">=":
if dep_version_int < low_check: if dep_version_int < low_check:
d.append({'name': dep[1], d.append({'name': dep[1],
'found': dep[0], 'found': dep[0],
"target": dep[2] + dep[3]}) "target": dep[2] + dep[3]})
continue continue
elif dep[2].strip() == ">": elif dep[2].strip() == ">":
if dep_version_int <= low_check: if dep_version_int <= low_check:
d.append({'name': dep[1], d.append({'name': dep[1],
'found': dep[0], 'found': dep[0],
"target": dep[2] + dep[3]}) "target": dep[2] + dep[3]})
continue continue
if dep[4] and dep[5]: if dep[4] and dep[5]:
if dep[4].strip() == "<": if dep[4].strip() == "<":

1743
cps/editbooks.py Normal file → Executable file
View File

File diff suppressed because it is too large Load Diff

127
cps/epub.py
View File

@ -20,24 +20,26 @@ import os
import zipfile import zipfile
from lxml import etree from lxml import etree
from . import isoLanguages from . import isoLanguages, cover
from .helper import split_authors from .helper import split_authors
from .constants import BookMeta from .constants import BookMeta
def _extract_cover(zip_file, cover_file, cover_path, tmp_file_name):
def extractCover(zipFile, coverFile, coverpath, tmp_file_name): if cover_file is None:
if coverFile is None:
return None return None
else: else:
zipCoverPath = os.path.join(coverpath, coverFile).replace('\\', '/') cf = extension = None
cf = zipFile.read(zipCoverPath) zip_cover_path = os.path.join(cover_path, cover_file).replace('\\', '/')
prefix = os.path.splitext(tmp_file_name)[0] prefix = os.path.splitext(tmp_file_name)[0]
tmp_cover_name = prefix + '.' + os.path.basename(zipCoverPath) tmp_cover_name = prefix + '.' + os.path.basename(zip_cover_path)
image = open(tmp_cover_name, 'wb') ext = os.path.splitext(tmp_cover_name)
image.write(cf) if len(ext) > 1:
image.close() extension = ext[1].lower()
return tmp_cover_name if extension in cover.COVER_EXTENSIONS:
cf = zip_file.read(zip_cover_path)
return cover.cover_processing(tmp_file_name, cf, extension)
def get_epub_info(tmp_file_path, original_file_name, original_file_extension): def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
@ -47,36 +49,44 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
'dc': 'http://purl.org/dc/elements/1.1/' 'dc': 'http://purl.org/dc/elements/1.1/'
} }
epubZip = zipfile.ZipFile(tmp_file_path) epub_zip = zipfile.ZipFile(tmp_file_path)
txt = epubZip.read('META-INF/container.xml') txt = epub_zip.read('META-INF/container.xml')
tree = etree.fromstring(txt) tree = etree.fromstring(txt)
cfname = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0] cf_name = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0]
cf = epubZip.read(cfname) cf = epub_zip.read(cf_name)
tree = etree.fromstring(cf) tree = etree.fromstring(cf)
coverpath = os.path.dirname(cfname) cover_path = os.path.dirname(cf_name)
p = tree.xpath('/pkg:package/pkg:metadata', namespaces=ns)[0] p = tree.xpath('/pkg:package/pkg:metadata', namespaces=ns)[0]
epub_metadata = {} epub_metadata = {}
for s in ['title', 'description', 'creator', 'language', 'subject']: for s in ['title', 'description', 'creator', 'language', 'subject', 'publisher', 'date']:
tmp = p.xpath('dc:%s/text()' % s, namespaces=ns) tmp = p.xpath('dc:%s/text()' % s, namespaces=ns)
if len(tmp) > 0: if len(tmp) > 0:
if s == 'creator': if s == 'creator':
epub_metadata[s] = ' & '.join(split_authors(tmp)) epub_metadata[s] = ' & '.join(split_authors(tmp))
elif s == 'subject': elif s == 'subject':
epub_metadata[s] = ', '.join(tmp) epub_metadata[s] = ', '.join(tmp)
elif s == 'date':
epub_metadata[s] = tmp[0][:10]
else: else:
epub_metadata[s] = tmp[0] epub_metadata[s] = tmp[0]
else: else:
epub_metadata[s] = u'Unknown' epub_metadata[s] = 'Unknown'
if epub_metadata['subject'] == u'Unknown': if epub_metadata['subject'] == 'Unknown':
epub_metadata['subject'] = '' epub_metadata['subject'] = ''
if epub_metadata['description'] == u'Unknown': if epub_metadata['publisher'] == 'Unknown':
epub_metadata['publisher'] = ''
if epub_metadata['date'] == 'Unknown':
epub_metadata['date'] = ''
if epub_metadata['description'] == 'Unknown':
description = tree.xpath("//*[local-name() = 'description']/text()") description = tree.xpath("//*[local-name() = 'description']/text()")
if len(description) > 0: if len(description) > 0:
epub_metadata['description'] = description epub_metadata['description'] = description
@ -86,9 +96,20 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
lang = epub_metadata['language'].split('-', 1)[0].lower() lang = epub_metadata['language'].split('-', 1)[0].lower()
epub_metadata['language'] = isoLanguages.get_lang3(lang) epub_metadata['language'] = isoLanguages.get_lang3(lang)
epub_metadata = parse_epbub_series(ns, tree, epub_metadata) epub_metadata = parse_epub_series(ns, tree, epub_metadata)
coverfile = parse_ebpub_cover(ns, tree, epubZip, coverpath, tmp_file_path) cover_file = parse_epub_cover(ns, tree, epub_zip, cover_path, tmp_file_path)
identifiers = []
for node in p.xpath('dc:identifier', namespaces=ns):
try:
identifier_name = node.attrib.values()[-1]
except IndexError:
continue
identifier_value = node.text
if identifier_name in ('uuid', 'calibre') or identifier_value is None:
continue
identifiers.append([identifier_name, identifier_value])
if not epub_metadata['title']: if not epub_metadata['title']:
title = original_file_name title = original_file_name
@ -100,45 +121,57 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
extension=original_file_extension, extension=original_file_extension,
title=title.encode('utf-8').decode('utf-8'), title=title.encode('utf-8').decode('utf-8'),
author=epub_metadata['creator'].encode('utf-8').decode('utf-8'), author=epub_metadata['creator'].encode('utf-8').decode('utf-8'),
cover=coverfile, cover=cover_file,
description=epub_metadata['description'], description=epub_metadata['description'],
tags=epub_metadata['subject'].encode('utf-8').decode('utf-8'), tags=epub_metadata['subject'].encode('utf-8').decode('utf-8'),
series=epub_metadata['series'].encode('utf-8').decode('utf-8'), series=epub_metadata['series'].encode('utf-8').decode('utf-8'),
series_id=epub_metadata['series_id'].encode('utf-8').decode('utf-8'), series_id=epub_metadata['series_id'].encode('utf-8').decode('utf-8'),
languages=epub_metadata['language'], languages=epub_metadata['language'],
publisher="") publisher=epub_metadata['publisher'].encode('utf-8').decode('utf-8'),
pubdate=epub_metadata['date'],
identifiers=identifiers)
def parse_ebpub_cover(ns, tree, epubZip, coverpath, tmp_file_path):
coversection = tree.xpath("/pkg:package/pkg:manifest/pkg:item[@id='cover-image']/@href", namespaces=ns) def parse_epub_cover(ns, tree, epub_zip, cover_path, tmp_file_path):
coverfile = None cover_section = tree.xpath("/pkg:package/pkg:manifest/pkg:item[@id='cover-image']/@href", namespaces=ns)
if len(coversection) > 0: cover_file = None
coverfile = extractCover(epubZip, coversection[0], coverpath, tmp_file_path) # if len(cover_section) > 0:
else: for cs in cover_section:
cover_file = _extract_cover(epub_zip, cs, cover_path, tmp_file_path)
if cover_file:
break
if not cover_file:
meta_cover = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='cover']/@content", namespaces=ns) meta_cover = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='cover']/@content", namespaces=ns)
if len(meta_cover) > 0: if len(meta_cover) > 0:
coversection = tree.xpath("/pkg:package/pkg:manifest/pkg:item[@id='"+meta_cover[0]+"']/@href", namespaces=ns) cover_section = tree.xpath(
"/pkg:package/pkg:manifest/pkg:item[@id='"+meta_cover[0]+"']/@href", namespaces=ns)
if not cover_section:
cover_section = tree.xpath(
"/pkg:package/pkg:manifest/pkg:item[@properties='" + meta_cover[0] + "']/@href", namespaces=ns)
else: else:
coversection = tree.xpath("/pkg:package/pkg:guide/pkg:reference/@href", namespaces=ns) cover_section = tree.xpath("/pkg:package/pkg:guide/pkg:reference/@href", namespaces=ns)
if len(coversection) > 0: for cs in cover_section:
filetype = coversection[0].rsplit('.', 1)[-1] filetype = cs.rsplit('.', 1)[-1]
if filetype == "xhtml" or filetype == "html": # if cover is (x)html format if filetype == "xhtml" or filetype == "html": # if cover is (x)html format
markup = epubZip.read(os.path.join(coverpath, coversection[0])) markup = epub_zip.read(os.path.join(cover_path, cs))
markupTree = etree.fromstring(markup) markup_tree = etree.fromstring(markup)
# no matter xhtml or html with no namespace # no matter xhtml or html with no namespace
imgsrc = markupTree.xpath("//*[local-name() = 'img']/@src") img_src = markup_tree.xpath("//*[local-name() = 'img']/@src")
# Alternative image source # Alternative image source
if not len(imgsrc): if not len(img_src):
imgsrc = markupTree.xpath("//attribute::*[contains(local-name(), 'href')]") img_src = markup_tree.xpath("//attribute::*[contains(local-name(), 'href')]")
if len(imgsrc): if len(img_src):
# imgsrc maybe startwith "../"" so fullpath join then relpath to cwd # img_src maybe start with "../"" so fullpath join then relpath to cwd
filename = os.path.relpath(os.path.join(os.path.dirname(os.path.join(coverpath, coversection[0])), filename = os.path.relpath(os.path.join(os.path.dirname(os.path.join(cover_path, cover_section[0])),
imgsrc[0])) img_src[0]))
coverfile = extractCover(epubZip, filename, "", tmp_file_path) cover_file = _extract_cover(epub_zip, filename, "", tmp_file_path)
else: else:
coverfile = extractCover(epubZip, coversection[0], coverpath, tmp_file_path) cover_file = _extract_cover(epub_zip, cs, cover_path, tmp_file_path)
return coverfile if cover_file: break
return cover_file
def parse_epbub_series(ns, tree, epub_metadata):
def parse_epub_series(ns, tree, epub_metadata):
series = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='calibre:series']/@content", namespaces=ns) series = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='calibre:series']/@content", namespaces=ns)
if len(series) > 0: if len(series) > 0:
epub_metadata['series'] = series[0] epub_metadata['series'] = series[0]

6
cps/error_handler.py
View File

@ -17,6 +17,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import traceback import traceback
from flask import render_template from flask import render_template
from werkzeug.exceptions import default_exceptions from werkzeug.exceptions import default_exceptions
try: try:
@ -42,8 +43,9 @@ def error_http(error):
def internal_error(error): def internal_error(error):
return render_template('http_error.html', return render_template('http_error.html',
error_code="Internal Server Error", error_code="500 Internal Server Error",
error_name=str(error), error_name='The server encountered an internal error and was unable to complete your '
'request. There is an error in the application.',
issue=True, issue=True,
unconfigured=False, unconfigured=False,
error_stack=traceback.format_exc().split("\n"), error_stack=traceback.format_exc().split("\n"),

18
cps/fb2.py
View File

@ -38,19 +38,19 @@ def get_fb2_info(tmp_file_path, original_file_extension):
if len(last_name): if len(last_name):
last_name = last_name[0] last_name = last_name[0]
else: else:
last_name = u'' last_name = ''
middle_name = element.xpath('fb:middle-name/text()', namespaces=ns) middle_name = element.xpath('fb:middle-name/text()', namespaces=ns)
if len(middle_name): if len(middle_name):
middle_name = middle_name[0] middle_name = middle_name[0]
else: else:
middle_name = u'' middle_name = ''
first_name = element.xpath('fb:first-name/text()', namespaces=ns) first_name = element.xpath('fb:first-name/text()', namespaces=ns)
if len(first_name): if len(first_name):
first_name = first_name[0] first_name = first_name[0]
else: else:
first_name = u'' first_name = ''
return (first_name + u' ' return (first_name + ' '
+ middle_name + u' ' + middle_name + ' '
+ last_name) + last_name)
author = str(", ".join(map(get_author, authors))) author = str(", ".join(map(get_author, authors)))
@ -59,12 +59,12 @@ def get_fb2_info(tmp_file_path, original_file_extension):
if len(title): if len(title):
title = str(title[0]) title = str(title[0])
else: else:
title = u'' title = ''
description = tree.xpath('/fb:FictionBook/fb:description/fb:publish-info/fb:book-name/text()', namespaces=ns) description = tree.xpath('/fb:FictionBook/fb:description/fb:publish-info/fb:book-name/text()', namespaces=ns)
if len(description): if len(description):
description = str(description[0]) description = str(description[0])
else: else:
description = u'' description = ''
return BookMeta( return BookMeta(
file_path=tmp_file_path, file_path=tmp_file_path,
@ -77,4 +77,6 @@ def get_fb2_info(tmp_file_path, original_file_extension):
series="", series="",
series_id="", series_id="",
languages="", languages="",
publisher="") publisher="",
pubdate="",
identifiers=[])

95
cps/fs.py Normal file
View File

@ -0,0 +1,95 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2020 mmonkey
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from . import logger
from .constants import CACHE_DIR
from os import makedirs, remove
from os.path import isdir, isfile, join
from shutil import rmtree
class FileSystem:
_instance = None
_cache_dir = CACHE_DIR
def __new__(cls):
if cls._instance is None:
cls._instance = super(FileSystem, cls).__new__(cls)
cls.log = logger.create()
return cls._instance
def get_cache_dir(self, cache_type=None):
if not isdir(self._cache_dir):
try:
makedirs(self._cache_dir)
except OSError:
self.log.info(f'Failed to create path {self._cache_dir} (Permission denied).')
raise
path = join(self._cache_dir, cache_type)
if cache_type and not isdir(path):
try:
makedirs(path)
except OSError:
self.log.info(f'Failed to create path {path} (Permission denied).')
raise
return path if cache_type else self._cache_dir
def get_cache_file_dir(self, filename, cache_type=None):
path = join(self.get_cache_dir(cache_type), filename[:2])
if not isdir(path):
try:
makedirs(path)
except OSError:
self.log.info(f'Failed to create path {path} (Permission denied).')
raise
return path
def get_cache_file_path(self, filename, cache_type=None):
return join(self.get_cache_file_dir(filename, cache_type), filename) if filename else None
def get_cache_file_exists(self, filename, cache_type=None):
path = self.get_cache_file_path(filename, cache_type)
return isfile(path)
def delete_cache_dir(self, cache_type=None):
if not cache_type and isdir(self._cache_dir):
try:
rmtree(self._cache_dir)
except OSError:
self.log.info(f'Failed to delete path {self._cache_dir} (Permission denied).')
raise
path = join(self._cache_dir, cache_type)
if cache_type and isdir(path):
try:
rmtree(path)
except OSError:
self.log.info(f'Failed to delete path {path} (Permission denied).')
raise
def delete_cache_file(self, filename, cache_type=None):
path = self.get_cache_file_path(filename, cache_type)
if isfile(path):
try:
remove(path)
except OSError:
self.log.info(f'Failed to delete path {path} (Permission denied).')
raise

12
cps/gdrive.py
View File

@ -55,7 +55,7 @@ def authenticate_google_drive():
try: try:
authUrl = gdriveutils.Gauth.Instance().auth.GetAuthUrl() authUrl = gdriveutils.Gauth.Instance().auth.GetAuthUrl()
except gdriveutils.InvalidConfigError: except gdriveutils.InvalidConfigError:
flash(_(u'Google Drive setup not completed, try to deactivate and activate Google Drive again'), flash(_('Google Drive setup not completed, try to deactivate and activate Google Drive again'),
category="error") category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
return redirect(authUrl) return redirect(authUrl)
@ -91,9 +91,9 @@ def watch_gdrive():
config.save() config.save()
except HttpError as e: except HttpError as e:
reason=json.loads(e.content)['error']['errors'][0] reason=json.loads(e.content)['error']['errors'][0]
if reason['reason'] == u'push.webhookUrlUnauthorized': if reason['reason'] == 'push.webhookUrlUnauthorized':
flash(_(u'Callback domain is not verified, ' flash(_('Callback domain is not verified, '
u'please follow steps to verify domain in google developer console'), category="error") 'please follow steps to verify domain in google developer console'), category="error")
else: else:
flash(reason['message'], category="error") flash(reason['message'], category="error")
@ -109,7 +109,7 @@ def revoke_watch_gdrive():
try: try:
gdriveutils.stopChannel(gdriveutils.Gdrive.Instance().drive, last_watch_response['id'], gdriveutils.stopChannel(gdriveutils.Gdrive.Instance().drive, last_watch_response['id'],
last_watch_response['resourceId']) last_watch_response['resourceId'])
except HttpError: except (HttpError, AttributeError):
pass pass
config.config_google_drive_watch_changes_response = {} config.config_google_drive_watch_changes_response = {}
config.save() config.save()
@ -152,7 +152,7 @@ try:
move(os.path.join(tmp_dir, "tmp_metadata.db"), dbpath) move(os.path.join(tmp_dir, "tmp_metadata.db"), dbpath)
calibre_db.reconnect_db(config, ub.app_DB_path) calibre_db.reconnect_db(config, ub.app_DB_path)
except Exception as ex: except Exception as ex:
log.debug_or_exception(ex) log.error_or_exception(ex)
return '' return ''
except AttributeError: except AttributeError:
pass pass

126
cps/gdriveutils.py
View File

@ -32,13 +32,10 @@ try:
from sqlalchemy.orm import declarative_base from sqlalchemy.orm import declarative_base
except ImportError: except ImportError:
from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.exc import OperationalError, InvalidRequestError from sqlalchemy.exc import OperationalError, InvalidRequestError, IntegrityError
from sqlalchemy.orm.exc import StaleDataError
from sqlalchemy.sql.expression import text from sqlalchemy.sql.expression import text
try:
from six import __version__ as six_version
except ImportError:
six_version = "not installed"
try: try:
from httplib2 import __version__ as httplib2_version from httplib2 import __version__ as httplib2_version
except ImportError: except ImportError:
@ -56,16 +53,18 @@ try:
from pydrive2.auth import GoogleAuth from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive from pydrive2.drive import GoogleDrive
from pydrive2.auth import RefreshError from pydrive2.auth import RefreshError
from pydrive2.files import ApiRequestError
except ImportError as err: except ImportError as err:
try: try:
from pydrive.auth import GoogleAuth from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive from pydrive.drive import GoogleDrive
from pydrive.auth import RefreshError from pydrive.auth import RefreshError
from pydrive.files import ApiRequestError
except ImportError as err: except ImportError as err:
importError = err importError = err
gdrive_support = False gdrive_support = False
from . import logger, cli, config from . import logger, cli_param, config
from .constants import CONFIG_DIR as _CONFIG_DIR from .constants import CONFIG_DIR as _CONFIG_DIR
@ -79,7 +78,7 @@ if gdrive_support:
if not logger.is_debug_enabled(): if not logger.is_debug_enabled():
logger.get('googleapiclient.discovery').setLevel(logger.logging.ERROR) logger.get('googleapiclient.discovery').setLevel(logger.logging.ERROR)
else: else:
log.debug("Cannot import pydrive,httplib2, using gdrive will not work: %s", importError) log.debug("Cannot import pydrive, httplib2, using gdrive will not work: {}".format(importError))
class Singleton: class Singleton:
@ -139,11 +138,12 @@ class Gdrive:
def __init__(self): def __init__(self):
self.drive = getDrive(gauth=Gauth.Instance().auth) self.drive = getDrive(gauth=Gauth.Instance().auth)
def is_gdrive_ready(): def is_gdrive_ready():
return os.path.exists(SETTINGS_YAML) and os.path.exists(CREDENTIALS) return os.path.exists(SETTINGS_YAML) and os.path.exists(CREDENTIALS)
engine = create_engine('sqlite:///{0}'.format(cli.gdpath), echo=False) engine = create_engine('sqlite:///{0}'.format(cli_param.gd_path), echo=False)
Base = declarative_base() Base = declarative_base()
# Open session for database connection # Open session for database connection
@ -191,10 +191,11 @@ def migrate():
session.execute('ALTER TABLE gdrive_ids2 RENAME to gdrive_ids') session.execute('ALTER TABLE gdrive_ids2 RENAME to gdrive_ids')
break break
if not os.path.exists(cli.gdpath): if not os.path.exists(cli_param.gd_path):
try: try:
Base.metadata.create_all(engine) Base.metadata.create_all(engine)
except Exception: except Exception as ex:
log.error("Error connect to database: {} - {}".format(cli_param.gd_path, ex))
raise raise
migrate() migrate()
@ -210,9 +211,9 @@ def getDrive(drive=None, gauth=None):
try: try:
gauth.Refresh() gauth.Refresh()
except RefreshError as e: except RefreshError as e:
log.error("Google Drive error: %s", e) log.error("Google Drive error: {}".format(e))
except Exception as ex: except Exception as ex:
log.debug_or_exception(ex) log.error_or_exception(ex)
else: else:
# Initialize the saved creds # Initialize the saved creds
gauth.Authorize() gauth.Authorize()
@ -222,7 +223,7 @@ def getDrive(drive=None, gauth=None):
try: try:
drive.auth.Refresh() drive.auth.Refresh()
except RefreshError as e: except RefreshError as e:
log.error("Google Drive error: %s", e) log.error("Google Drive error: {}".format(e))
return drive return drive
def listRootFolders(): def listRootFolders():
@ -231,7 +232,7 @@ def listRootFolders():
folder = "'root' in parents and mimeType = 'application/vnd.google-apps.folder' and trashed = false" folder = "'root' in parents and mimeType = 'application/vnd.google-apps.folder' and trashed = false"
fileList = drive.ListFile({'q': folder}).GetList() fileList = drive.ListFile({'q': folder}).GetList()
except (ServerNotFoundError, ssl.SSLError, RefreshError) as e: except (ServerNotFoundError, ssl.SSLError, RefreshError) as e:
log.info("GDrive Error %s" % e) log.info("GDrive Error {}".format(e))
fileList = [] fileList = []
return fileList return fileList
@ -269,8 +270,7 @@ def getEbooksFolderId(drive=None):
try: try:
session.commit() session.commit()
except OperationalError as ex: except OperationalError as ex:
log.error("gdrive.db DB is not Writeable") log.error_or_exception('Database error: {}'.format(ex))
log.debug('Database error: %s', ex)
session.rollback() session.rollback()
return gDriveId.gdrive_id return gDriveId.gdrive_id
@ -286,6 +286,7 @@ def getFile(pathId, fileName, drive):
def getFolderId(path, drive): def getFolderId(path, drive):
# drive = getDrive(drive) # drive = getDrive(drive)
currentFolderId = None
try: try:
currentFolderId = getEbooksFolderId(drive) currentFolderId = getEbooksFolderId(drive)
sqlCheckPath = path if path[-1] == '/' else path + '/' sqlCheckPath = path if path[-1] == '/' else path + '/'
@ -318,10 +319,14 @@ def getFolderId(path, drive):
session.commit() session.commit()
else: else:
currentFolderId = storedPathName.gdrive_id currentFolderId = storedPathName.gdrive_id
except OperationalError as ex: except (OperationalError, IntegrityError, StaleDataError) as ex:
log.error("gdrive.db DB is not Writeable") log.error_or_exception('Database error: {}'.format(ex))
log.debug('Database error: %s', ex)
session.rollback() session.rollback()
except ApiRequestError as ex:
log.error('{} {}'.format(ex.error['message'], path))
session.rollback()
except RefreshError as ex:
log.error(ex)
return currentFolderId return currentFolderId
@ -355,16 +360,27 @@ def moveGdriveFolderRemote(origin_file, target_folder):
children = drive.auth.service.children().list(folderId=previous_parents).execute() children = drive.auth.service.children().list(folderId=previous_parents).execute()
gFileTargetDir = getFileFromEbooksFolder(None, target_folder) gFileTargetDir = getFileFromEbooksFolder(None, target_folder)
if not gFileTargetDir: if not gFileTargetDir:
# Folder is not existing, create, and move folder
gFileTargetDir = drive.CreateFile( gFileTargetDir = drive.CreateFile(
{'title': target_folder, 'parents': [{"kind": "drive#fileLink", 'id': getEbooksFolderId()}], {'title': target_folder, 'parents': [{"kind": "drive#fileLink", 'id': getEbooksFolderId()}],
"mimeType": "application/vnd.google-apps.folder"}) "mimeType": "application/vnd.google-apps.folder"})
gFileTargetDir.Upload() gFileTargetDir.Upload()
# Move the file to the new folder # Move the file to the new folder
drive.auth.service.files().update(fileId=origin_file['id'], drive.auth.service.files().update(fileId=origin_file['id'],
addParents=gFileTargetDir['id'], addParents=gFileTargetDir['id'],
removeParents=previous_parents, removeParents=previous_parents,
fields='id, parents').execute() fields='id, parents').execute()
elif gFileTargetDir['title'] != target_folder:
# Folder is not existing, create, and move folder
drive.auth.service.files().patch(fileId=origin_file['id'],
body={'title': target_folder},
fields='title').execute()
else:
# Move the file to the new folder
drive.auth.service.files().update(fileId=origin_file['id'],
addParents=gFileTargetDir['id'],
removeParents=previous_parents,
fields='id, parents').execute()
# if previous_parents has no children anymore, delete original fileparent # if previous_parents has no children anymore, delete original fileparent
if len(children['items']) == 1: if len(children['items']) == 1:
deleteDatabaseEntry(previous_parents) deleteDatabaseEntry(previous_parents)
@ -412,24 +428,24 @@ def uploadFileToEbooksFolder(destFile, f):
splitDir = destFile.split('/') splitDir = destFile.split('/')
for i, x in enumerate(splitDir): for i, x in enumerate(splitDir):
if i == len(splitDir)-1: if i == len(splitDir)-1:
existingFiles = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" % existing_Files = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" %
(x.replace("'", r"\'"), parent['id'])}).GetList() (x.replace("'", r"\'"), parent['id'])}).GetList()
if len(existingFiles) > 0: if len(existing_Files) > 0:
driveFile = existingFiles[0] driveFile = existing_Files[0]
else: else:
driveFile = drive.CreateFile({'title': x, driveFile = drive.CreateFile({'title': x,
'parents': [{"kind": "drive#fileLink", 'id': parent['id']}], }) 'parents': [{"kind": "drive#fileLink", 'id': parent['id']}], })
driveFile.SetContentFile(f) driveFile.SetContentFile(f)
driveFile.Upload() driveFile.Upload()
else: else:
existingFolder = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" % existing_Folder = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" %
(x.replace("'", r"\'"), parent['id'])}).GetList() (x.replace("'", r"\'"), parent['id'])}).GetList()
if len(existingFolder) == 0: if len(existing_Folder) == 0:
parent = drive.CreateFile({'title': x, 'parents': [{"kind": "drive#fileLink", 'id': parent['id']}], parent = drive.CreateFile({'title': x, 'parents': [{"kind": "drive#fileLink", 'id': parent['id']}],
"mimeType": "application/vnd.google-apps.folder"}) "mimeType": "application/vnd.google-apps.folder"})
parent.Upload() parent.Upload()
else: else:
parent = existingFolder[0] parent = existing_Folder[0]
def watchChange(drive, channel_id, channel_type, channel_address, def watchChange(drive, channel_id, channel_type, channel_address,
@ -528,8 +544,8 @@ def deleteDatabaseOnChange():
session.commit() session.commit()
except (OperationalError, InvalidRequestError) as ex: except (OperationalError, InvalidRequestError) as ex:
session.rollback() session.rollback()
log.debug('Database error: %s', ex) log.error_or_exception('Database error: {}'.format(ex))
log.error(u"GDrive DB is not Writeable") session.rollback()
def updateGdriveCalibreFromLocal(): def updateGdriveCalibreFromLocal():
@ -540,15 +556,14 @@ def updateGdriveCalibreFromLocal():
# update gdrive.db on edit of books title # update gdrive.db on edit of books title
def updateDatabaseOnEdit(ID,newPath): def updateDatabaseOnEdit(ID,newPath):
sqlCheckPath = newPath if newPath[-1] == '/' else newPath + u'/' sqlCheckPath = newPath if newPath[-1] == '/' else newPath + '/'
storedPathName = session.query(GdriveId).filter(GdriveId.gdrive_id == ID).first() storedPathName = session.query(GdriveId).filter(GdriveId.gdrive_id == ID).first()
if storedPathName: if storedPathName:
storedPathName.path = sqlCheckPath storedPathName.path = sqlCheckPath
try: try:
session.commit() session.commit()
except OperationalError as ex: except OperationalError as ex:
log.error("gdrive.db DB is not Writeable") log.error_or_exception('Database error: {}'.format(ex))
log.debug('Database error: %s', ex)
session.rollback() session.rollback()
@ -558,12 +573,12 @@ def deleteDatabaseEntry(ID):
try: try:
session.commit() session.commit()
except OperationalError as ex: except OperationalError as ex:
log.error("gdrive.db DB is not Writeable") log.error_or_exception('Database error: {}'.format(ex))
log.debug('Database error: %s', ex)
session.rollback() session.rollback()
# Gets cover file from gdrive # Gets cover file from gdrive
# ToDo: Check is this right everyone get read permissions on cover files?
def get_cover_via_gdrive(cover_path): def get_cover_via_gdrive(cover_path):
df = getFileFromEbooksFolder(cover_path, 'cover.jpg') df = getFileFromEbooksFolder(cover_path, 'cover.jpg')
if df: if df:
@ -580,8 +595,30 @@ def get_cover_via_gdrive(cover_path):
try: try:
session.commit() session.commit()
except OperationalError as ex: except OperationalError as ex:
log.error("gdrive.db DB is not Writeable") log.error_or_exception('Database error: {}'.format(ex))
log.debug('Database error: %s', ex) session.rollback()
return df.metadata.get('webContentLink')
else:
return None
# Gets cover file from gdrive
def get_metadata_backup_via_gdrive(metadata_path):
df = getFileFromEbooksFolder(metadata_path, 'metadata.opf')
if df:
if not session.query(PermissionAdded).filter(PermissionAdded.gdrive_id == df['id']).first():
df.GetPermissions()
df.InsertPermission({
'type': 'anyone',
'value': 'anyone',
'role': 'writer', # ToDo needs write access
'withLink': True})
permissionAdded = PermissionAdded()
permissionAdded.gdrive_id = df['id']
session.add(permissionAdded)
try:
session.commit()
except OperationalError as ex:
log.error_or_exception('Database error: {}'.format(ex))
session.rollback() session.rollback()
return df.metadata.get('webContentLink') return df.metadata.get('webContentLink')
else: else:
@ -603,7 +640,7 @@ def do_gdrive_download(df, headers, convert_encoding=False):
def stream(convert_encoding): def stream(convert_encoding):
for byte in s: for byte in s:
headers = {"Range": 'bytes=%s-%s' % (byte[0], byte[1])} headers = {"Range": 'bytes={}-{}'.format(byte[0], byte[1])}
resp, content = df.auth.Get_Http_Object().request(download_url, headers=headers) resp, content = df.auth.Get_Http_Object().request(download_url, headers=headers)
if resp.status == 206: if resp.status == 206:
if convert_encoding: if convert_encoding:
@ -611,7 +648,7 @@ def do_gdrive_download(df, headers, convert_encoding=False):
content = content.decode(result['encoding']).encode('utf-8') content = content.decode(result['encoding']).encode('utf-8')
yield content yield content
else: else:
log.warning('An error occurred: %s', resp) log.warning('An error occurred: {}'.format(resp))
return return
return Response(stream_with_context(stream(convert_encoding)), headers=headers) return Response(stream_with_context(stream(convert_encoding)), headers=headers)
@ -668,8 +705,3 @@ def get_error_text(client_secrets=None):
return 'Callback url (redirect url) is missing in client_secrets.json' return 'Callback url (redirect url) is missing in client_secrets.json'
if client_secrets: if client_secrets:
client_secrets.update(filedata['web']) client_secrets.update(filedata['web'])
def get_versions():
return {'six': six_version,
'httplib2': httplib2_version}

29
cps/gevent_wsgi.py Normal file
View File

@ -0,0 +1,29 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from gevent.pywsgi import WSGIHandler
class MyWSGIHandler(WSGIHandler):
def get_environ(self):
env = super().get_environ()
path, __ = self.path.split('?', 1) if '?' in self.path else (self.path, '')
env['RAW_URI'] = path
return env

867
cps/helper.py
View File

File diff suppressed because it is too large Load Diff

3
cps/isoLanguages.py
View File

@ -49,7 +49,7 @@ except ImportError:
def get_language_names(locale): def get_language_names(locale):
return _LANGUAGE_NAMES.get(locale) return _LANGUAGE_NAMES.get(str(locale))
def get_language_name(locale, lang_code): def get_language_name(locale, lang_code):
@ -57,6 +57,7 @@ def get_language_name(locale, lang_code):
return get_language_names(locale)[lang_code] return get_language_names(locale)[lang_code]
except KeyError: except KeyError:
log.error('Missing translation for language name: {}'.format(lang_code)) log.error('Missing translation for language name: {}'.format(lang_code))
return "Unknown"
def get_language_codes(locale, language_names, remainder=None): def get_language_codes(locale, language_names, remainder=None):

1971
cps/iso_language_names.py
View File

File diff suppressed because it is too large Load Diff

53
cps/jinjia.py
View File

@ -22,17 +22,17 @@
# custom jinja filters # custom jinja filters
from markupsafe import escape
import datetime import datetime
import mimetypes import mimetypes
from uuid import uuid4 from uuid import uuid4
from babel.dates import format_date # from babel.dates import format_date
from flask import Blueprint, request, url_for from flask import Blueprint, request, url_for
from flask_babel import get_locale from flask_babel import format_date
from flask_login import current_user from flask_login import current_user
from markupsafe import escape
from . import logger
from . import constants, logger
jinjia = Blueprint('jinjia', __name__) jinjia = Blueprint('jinjia', __name__)
log = logger.create() log = logger.create()
@ -77,7 +77,7 @@ def mimetype_filter(val):
@jinjia.app_template_filter('formatdate') @jinjia.app_template_filter('formatdate')
def formatdate_filter(val): def formatdate_filter(val):
try: try:
return format_date(val, format='medium', locale=get_locale()) return format_date(val, format='medium')
except AttributeError as e: except AttributeError as e:
log.error('Babel error: %s, Current user locale: %s, Current User: %s', e, log.error('Babel error: %s, Current user locale: %s, Current User: %s', e,
current_user.locale, current_user.locale,
@ -128,12 +128,55 @@ def formatseriesindex_filter(series_index):
return series_index return series_index
return 0 return 0
@jinjia.app_template_filter('escapedlink') @jinjia.app_template_filter('escapedlink')
def escapedlink_filter(url, text): def escapedlink_filter(url, text):
return "<a href='{}'>{}</a>".format(url, escape(text)) return "<a href='{}'>{}</a>".format(url, escape(text))
@jinjia.app_template_filter('uuidfilter') @jinjia.app_template_filter('uuidfilter')
def uuidfilter(var): def uuidfilter(var):
return uuid4() return uuid4()
@jinjia.app_template_filter('cache_timestamp')
def cache_timestamp(rolling_period='month'):
if rolling_period == 'day':
return str(int(datetime.datetime.today().replace(hour=1, minute=1).timestamp()))
elif rolling_period == 'year':
return str(int(datetime.datetime.today().replace(day=1).timestamp()))
else:
return str(int(datetime.datetime.today().replace(month=1, day=1).timestamp()))
@jinjia.app_template_filter('last_modified')
def book_last_modified(book):
return str(int(book.last_modified.timestamp()))
@jinjia.app_template_filter('get_cover_srcset')
def get_cover_srcset(book):
srcset = list()
resolutions = {
constants.COVER_THUMBNAIL_SMALL: 'sm',
constants.COVER_THUMBNAIL_MEDIUM: 'md',
constants.COVER_THUMBNAIL_LARGE: 'lg'
}
for resolution, shortname in resolutions.items():
url = url_for('web.get_cover', book_id=book.id, resolution=shortname, c=book_last_modified(book))
srcset.append(f'{url} {resolution}x')
return ', '.join(srcset)
@jinjia.app_template_filter('get_series_srcset')
def get_cover_srcset(series):
srcset = list()
resolutions = {
constants.COVER_THUMBNAIL_SMALL: 'sm',
constants.COVER_THUMBNAIL_MEDIUM: 'md',
constants.COVER_THUMBNAIL_LARGE: 'lg'
}
for resolution, shortname in resolutions.items():
url = url_for('web.get_series_cover', series_id=series.id, resolution=shortname, c=cache_timestamp())
srcset.append(f'{url} {resolution}x')
return ', '.join(srcset)

242
cps/kobo.py
View File

@ -22,11 +22,8 @@ import datetime
import os import os
import uuid import uuid
from time import gmtime, strftime from time import gmtime, strftime
import json
try: from urllib.parse import unquote
from urllib import unquote
except ImportError:
from urllib.parse import unquote
from flask import ( from flask import (
Blueprint, Blueprint,
@ -48,7 +45,8 @@ import requests
from . import config, logger, kobo_auth, db, calibre_db, helper, shelf as shelf_lib, ub, csrf, kobo_sync_status from . import config, logger, kobo_auth, db, calibre_db, helper, shelf as shelf_lib, ub, csrf, kobo_sync_status
from .constants import sqlalchemy_version2 from . import isoLanguages
from .constants import sqlalchemy_version2, COVER_THUMBNAIL_SMALL
from .helper import get_download_link from .helper import get_download_link
from .services import SyncToken as SyncToken from .services import SyncToken as SyncToken
from .web import download_required from .web import download_required
@ -102,6 +100,8 @@ def make_request_to_kobo_store(sync_token=None):
allow_redirects=False, allow_redirects=False,
timeout=(2, 10) timeout=(2, 10)
) )
log.debug("Content: " + str(store_response.content))
log.debug("StatusCode: " + str(store_response.status_code))
return store_response return store_response
@ -110,7 +110,8 @@ def redirect_or_proxy_request():
if request.method == "GET": if request.method == "GET":
return redirect(get_store_url_for_current_request(), 307) return redirect(get_store_url_for_current_request(), 307)
else: else:
# The Kobo device turns other request types into GET requests on redirects, so we instead proxy to the Kobo store ourselves. # The Kobo device turns other request types into GET requests on redirects,
# so we instead proxy to the Kobo store ourselves.
store_response = make_request_to_kobo_store() store_response = make_request_to_kobo_store()
response_headers = store_response.headers response_headers = store_response.headers
@ -129,7 +130,7 @@ def convert_to_kobo_timestamp_string(timestamp):
return timestamp.strftime("%Y-%m-%dT%H:%M:%SZ") return timestamp.strftime("%Y-%m-%dT%H:%M:%SZ")
except AttributeError as exc: except AttributeError as exc:
log.debug("Timestamp not valid: {}".format(exc)) log.debug("Timestamp not valid: {}".format(exc))
return datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ") return datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
@kobo.route("/v1/library/sync") @kobo.route("/v1/library/sync")
@ -142,16 +143,20 @@ def HandleSyncRequest():
if not current_app.wsgi_app.is_proxied: if not current_app.wsgi_app.is_proxied:
log.debug('Kobo: Received unproxied request, changed request port to external server port') log.debug('Kobo: Received unproxied request, changed request port to external server port')
# TODO: Limit the number of books return per sync call, and rely on the sync-continuatation header # if no books synced don't respect sync_token
# instead so that the device triggers another sync. if not ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.user_id == current_user.id).count():
sync_token.books_last_modified = datetime.datetime.min
sync_token.books_last_created = datetime.datetime.min
sync_token.reading_state_last_modified = datetime.datetime.min
new_books_last_modified = sync_token.books_last_modified new_books_last_modified = sync_token.books_last_modified # needed for sync selected shelfs only
new_books_last_created = sync_token.books_last_created new_books_last_created = sync_token.books_last_created # needed to distinguish between new and changed entitlement
new_reading_state_last_modified = sync_token.reading_state_last_modified new_reading_state_last_modified = sync_token.reading_state_last_modified
new_archived_last_modified = datetime.datetime.min new_archived_last_modified = datetime.datetime.min
sync_results = [] sync_results = []
# We reload the book database so that the user get's a fresh view of the library # We reload the book database so that the user gets a fresh view of the library
# in case of external changes (e.g: adding a book through Calibre). # in case of external changes (e.g: adding a book through Calibre).
calibre_db.reconnect_db(config, ub.app_DB_path) calibre_db.reconnect_db(config, ub.app_DB_path)
@ -169,21 +174,20 @@ def HandleSyncRequest():
ub.BookShelf.date_added, ub.BookShelf.date_added,
ub.ArchivedBook.is_archived) ub.ArchivedBook.is_archived)
changed_entries = (changed_entries changed_entries = (changed_entries
.join(db.Data).outerjoin(ub.ArchivedBook, db.Books.id == ub.ArchivedBook.book_id) .join(db.Data).outerjoin(ub.ArchivedBook, and_(db.Books.id == ub.ArchivedBook.book_id,
.join(ub.KoboSyncedBooks, ub.KoboSyncedBooks.book_id == db.Books.id, isouter=True) ub.ArchivedBook.user_id == current_user.id))
.filter(or_(ub.KoboSyncedBooks.user_id != current_user.id, .filter(db.Books.id.notin_(calibre_db.session.query(ub.KoboSyncedBooks.book_id)
ub.KoboSyncedBooks.book_id == None)) .filter(ub.KoboSyncedBooks.user_id == current_user.id)))
.filter(ub.BookShelf.date_added > sync_token.books_last_modified) .filter(ub.BookShelf.date_added > sync_token.books_last_modified)
.filter(db.Data.format.in_(KOBO_FORMATS)) .filter(db.Data.format.in_(KOBO_FORMATS))
.filter(calibre_db.common_filters(allow_show_archived=True)) .filter(calibre_db.common_filters(allow_show_archived=True))
.order_by(db.Books.id) .order_by(db.Books.id)
.order_by(ub.ArchivedBook.last_modified) .order_by(ub.ArchivedBook.last_modified)
.join(ub.BookShelf, db.Books.id == ub.BookShelf.book_id) .join(ub.BookShelf, db.Books.id == ub.BookShelf.book_id)
.join(ub.Shelf) .join(ub.Shelf)
.filter(ub.Shelf.user_id == current_user.id) .filter(ub.Shelf.user_id == current_user.id)
.filter(ub.Shelf.kobo_sync) .filter(ub.Shelf.kobo_sync)
.distinct() .distinct())
)
else: else:
if sqlalchemy_version2: if sqlalchemy_version2:
changed_entries = select(db.Books, ub.ArchivedBook.last_modified, ub.ArchivedBook.is_archived) changed_entries = select(db.Books, ub.ArchivedBook.last_modified, ub.ArchivedBook.is_archived)
@ -192,26 +196,24 @@ def HandleSyncRequest():
ub.ArchivedBook.last_modified, ub.ArchivedBook.last_modified,
ub.ArchivedBook.is_archived) ub.ArchivedBook.is_archived)
changed_entries = (changed_entries changed_entries = (changed_entries
.join(db.Data).outerjoin(ub.ArchivedBook, db.Books.id == ub.ArchivedBook.book_id) .join(db.Data).outerjoin(ub.ArchivedBook, and_(db.Books.id == ub.ArchivedBook.book_id,
.join(ub.KoboSyncedBooks, ub.KoboSyncedBooks.book_id == db.Books.id, isouter=True) ub.ArchivedBook.user_id == current_user.id))
.filter(or_(ub.KoboSyncedBooks.user_id != current_user.id, .filter(db.Books.id.notin_(calibre_db.session.query(ub.KoboSyncedBooks.book_id)
ub.KoboSyncedBooks.book_id == None)) .filter(ub.KoboSyncedBooks.user_id == current_user.id)))
.filter(calibre_db.common_filters()) .filter(calibre_db.common_filters(allow_show_archived=True))
.filter(db.Data.format.in_(KOBO_FORMATS)) .filter(db.Data.format.in_(KOBO_FORMATS))
.order_by(db.Books.last_modified) .order_by(db.Books.last_modified)
.order_by(db.Books.id) .order_by(db.Books.id))
)
reading_states_in_new_entitlements = [] reading_states_in_new_entitlements = []
if sqlalchemy_version2: if sqlalchemy_version2:
books = calibre_db.session.execute(changed_entries.limit(SYNC_ITEM_LIMIT)) books = calibre_db.session.execute(changed_entries.limit(SYNC_ITEM_LIMIT))
else: else:
books = changed_entries.limit(SYNC_ITEM_LIMIT) books = changed_entries.limit(SYNC_ITEM_LIMIT)
log.debug("Books to Sync: {}".format(len(books.all())))
for book in books: for book in books:
kobo_sync_status.add_synced_books(book.Books.id)
formats = [data.format for data in book.Books.data] formats = [data.format for data in book.Books.data]
if not 'KEPUB' in formats and config.config_kepubifypath and 'EPUB' in formats: if 'KEPUB' not in formats and config.config_kepubifypath and 'EPUB' in formats:
helper.convert_book_format(book.Books.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name) helper.convert_book_format(book.Books.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
kobo_reading_state = get_or_create_reading_state(book.Books.id) kobo_reading_state = get_or_create_reading_state(book.Books.id)
@ -248,14 +250,17 @@ def HandleSyncRequest():
pass pass
new_books_last_created = max(ts_created, new_books_last_created) new_books_last_created = max(ts_created, new_books_last_created)
kobo_sync_status.add_synced_books(book.Books.id)
if sqlalchemy_version2: if sqlalchemy_version2:
max_change = calibre_db.session.execute(changed_entries max_change = calibre_db.session.execute(changed_entries
.filter(ub.ArchivedBook.is_archived) .filter(ub.ArchivedBook.is_archived)
.filter(ub.ArchivedBook.user_id == current_user.id)
.order_by(func.datetime(ub.ArchivedBook.last_modified).desc()))\ .order_by(func.datetime(ub.ArchivedBook.last_modified).desc()))\
.columns(db.Books).first() .columns(db.Books).first()
else: else:
max_change = changed_entries.from_self().filter(ub.ArchivedBook.is_archived) \ max_change = changed_entries.from_self().filter(ub.ArchivedBook.is_archived)\
.filter(ub.ArchivedBook.user_id == current_user.id) \
.order_by(func.datetime(ub.ArchivedBook.last_modified).desc()).first() .order_by(func.datetime(ub.ArchivedBook.last_modified).desc()).first()
max_change = max_change.last_modified if max_change else new_archived_last_modified max_change = max_change.last_modified if max_change else new_archived_last_modified
@ -290,7 +295,8 @@ def HandleSyncRequest():
changed_reading_states = changed_reading_states.filter( changed_reading_states = changed_reading_states.filter(
and_(ub.KoboReadingState.user_id == current_user.id, and_(ub.KoboReadingState.user_id == current_user.id,
ub.KoboReadingState.book_id.notin_(reading_states_in_new_entitlements))) ub.KoboReadingState.book_id.notin_(reading_states_in_new_entitlements)))\
.order_by(ub.KoboReadingState.last_modified)
cont_sync |= bool(changed_reading_states.count() > SYNC_ITEM_LIMIT) cont_sync |= bool(changed_reading_states.count() > SYNC_ITEM_LIMIT)
for kobo_reading_state in changed_reading_states.limit(SYNC_ITEM_LIMIT).all(): for kobo_reading_state in changed_reading_states.limit(SYNC_ITEM_LIMIT).all():
book = calibre_db.session.query(db.Books).filter(db.Books.id == kobo_reading_state.book_id).one_or_none() book = calibre_db.session.query(db.Books).filter(db.Books.id == kobo_reading_state.book_id).one_or_none()
@ -304,18 +310,19 @@ def HandleSyncRequest():
sync_shelves(sync_token, sync_results, only_kobo_shelves) sync_shelves(sync_token, sync_results, only_kobo_shelves)
sync_token.books_last_created = new_books_last_created # update last created timestamp to distinguish between new and changed entitlements
if not cont_sync:
sync_token.books_last_created = new_books_last_created
sync_token.books_last_modified = new_books_last_modified sync_token.books_last_modified = new_books_last_modified
sync_token.archive_last_modified = new_archived_last_modified sync_token.archive_last_modified = new_archived_last_modified
sync_token.reading_state_last_modified = new_reading_state_last_modified sync_token.reading_state_last_modified = new_reading_state_last_modified
# sync_token.books_last_id = books_last_id
return generate_sync_response(sync_token, sync_results, cont_sync) return generate_sync_response(sync_token, sync_results, cont_sync)
def generate_sync_response(sync_token, sync_results, set_cont=False): def generate_sync_response(sync_token, sync_results, set_cont=False):
extra_headers = {} extra_headers = {}
if config.config_kobo_proxy: if config.config_kobo_proxy and not set_cont:
# Merge in sync results from the official Kobo store. # Merge in sync results from the official Kobo store.
try: try:
store_response = make_request_to_kobo_store(sync_token) store_response = make_request_to_kobo_store(sync_token)
@ -334,8 +341,9 @@ def generate_sync_response(sync_token, sync_results, set_cont=False):
sync_token.to_headers(extra_headers) sync_token.to_headers(extra_headers)
# log.debug("Kobo Sync Content: {}".format(sync_results)) # log.debug("Kobo Sync Content: {}".format(sync_results))
response = make_response(jsonify(sync_results), extra_headers) # jsonify decodes the unicode string different to what kobo expects
response = make_response(json.dumps(sync_results), extra_headers)
response.headers["Content-Type"] = "application/json; charset=utf-8"
return response return response
@ -348,11 +356,13 @@ def HandleMetadataRequest(book_uuid):
log.info("Kobo library metadata request received for book %s" % book_uuid) log.info("Kobo library metadata request received for book %s" % book_uuid)
book = calibre_db.get_book_by_uuid(book_uuid) book = calibre_db.get_book_by_uuid(book_uuid)
if not book or not book.data: if not book or not book.data:
log.info(u"Book %s not found in database", book_uuid) log.info("Book %s not found in database", book_uuid)
return redirect_or_proxy_request() return redirect_or_proxy_request()
metadata = get_metadata(book) metadata = get_metadata(book)
return jsonify([metadata]) response = make_response(json.dumps([metadata], ensure_ascii=False))
response.headers["Content-Type"] = "application/json; charset=utf-8"
return response
def get_download_url_for_book(book, book_format): def get_download_url_for_book(book, book_format):
@ -380,10 +390,10 @@ def get_download_url_for_book(book, book_format):
def create_book_entitlement(book, archived): def create_book_entitlement(book, archived):
book_uuid = book.uuid book_uuid = str(book.uuid)
return { return {
"Accessibility": "Full", "Accessibility": "Full",
"ActivePeriod": {"From": convert_to_kobo_timestamp_string(datetime.datetime.now())}, "ActivePeriod": {"From": convert_to_kobo_timestamp_string(datetime.datetime.utcnow())},
"Created": convert_to_kobo_timestamp_string(book.timestamp), "Created": convert_to_kobo_timestamp_string(book.timestamp),
"CrossRevisionId": book_uuid, "CrossRevisionId": book_uuid,
"Id": book_uuid, "Id": book_uuid,
@ -407,18 +417,15 @@ def get_description(book):
return book.comments[0].text return book.comments[0].text
# TODO handle multiple authors
def get_author(book): def get_author(book):
if not book.authors: if not book.authors:
return {"Contributors": None} return {"Contributors": None}
if len(book.authors) > 1: author_list = []
author_list = [] autor_roles = []
autor_roles = [] for author in book.authors:
for author in book.authors: autor_roles.append({"Name": author.name})
autor_roles.append({"Name":author.name, "Role":"Author"}) author_list.append(author.name)
author_list.append(author.name) return {"ContributorRoles": autor_roles, "Contributors": author_list}
return {"ContributorRoles": autor_roles, "Contributors":author_list}
return {"ContributorRoles": [{"Name":book.authors[0].name, "Role":"Author"}], "Contributors": book.authors[0].name}
def get_publisher(book): def get_publisher(book):
@ -432,10 +439,17 @@ def get_series(book):
return None return None
return book.series[0].name return book.series[0].name
def get_seriesindex(book): def get_seriesindex(book):
return book.series_index or 1 return book.series_index or 1
def get_language(book):
if not book.languages:
return 'en'
return isoLanguages.get(part3=book.languages[0].lang_code).part1
def get_metadata(book): def get_metadata(book):
download_urls = [] download_urls = []
kepub = [data for data in book.data if data.format == 'KEPUB'] kepub = [data for data in book.data if data.format == 'KEPUB']
@ -473,12 +487,10 @@ def get_metadata(book):
"IsInternetArchive": False, "IsInternetArchive": False,
"IsPreOrder": False, "IsPreOrder": False,
"IsSocialEnabled": True, "IsSocialEnabled": True,
"Language": "en", "Language": get_language(book),
"PhoneticPronunciations": {}, "PhoneticPronunciations": {},
# TODO: Fix book.pubdate to return a datetime object so that we can easily "PublicationDate": convert_to_kobo_timestamp_string(book.pubdate),
# convert it to the format Kobo devices expect. "Publisher": {"Imprint": "", "Name": get_publisher(book), },
"PublicationDate": book.pubdate,
"Publisher": {"Imprint": "", "Name": get_publisher(book),},
"RevisionId": book_uuid, "RevisionId": book_uuid,
"Title": book.title, "Title": book.title,
"WorkId": book_uuid, "WorkId": book_uuid,
@ -492,17 +504,18 @@ def get_metadata(book):
"Number": get_seriesindex(book), # ToDo Check int() ? "Number": get_seriesindex(book), # ToDo Check int() ?
"NumberFloat": float(get_seriesindex(book)), "NumberFloat": float(get_seriesindex(book)),
# Get a deterministic id based on the series name. # Get a deterministic id based on the series name.
"Id": uuid.uuid3(uuid.NAMESPACE_DNS, name), "Id": str(uuid.uuid3(uuid.NAMESPACE_DNS, name)),
} }
return metadata return metadata
@csrf.exempt @csrf.exempt
@kobo.route("/v1/library/tags", methods=["POST", "DELETE"]) @kobo.route("/v1/library/tags", methods=["POST", "DELETE"])
@requires_kobo_auth @requires_kobo_auth
# Creates a Shelf with the given items, and returns the shelf's uuid. # Creates a Shelf with the given items, and returns the shelf's uuid.
def HandleTagCreate(): def HandleTagCreate():
# catch delete requests, otherwise the are handeld in the book delete handler # catch delete requests, otherwise the are handled in the book delete handler
if request.method == "DELETE": if request.method == "DELETE":
abort(405) abort(405)
name, items = None, None name, items = None, None
@ -545,11 +558,9 @@ def HandleTagUpdate(tag_id):
else: else:
abort(404, description="Collection isn't known to CalibreWeb") abort(404, description="Collection isn't known to CalibreWeb")
if not shelf_lib.check_shelf_edit_permissions(shelf):
abort(401, description="User is unauthaurized to edit shelf.")
if request.method == "DELETE": if request.method == "DELETE":
shelf_lib.delete_shelf_helper(shelf) if not shelf_lib.delete_shelf_helper(shelf):
abort(401, description="Error deleting Shelf")
else: else:
name = None name = None
try: try:
@ -667,11 +678,8 @@ def HandleTagRemoveItem(tag_id):
# Note: Public shelves that aren't owned by the user aren't supported. # Note: Public shelves that aren't owned by the user aren't supported.
def sync_shelves(sync_token, sync_results, only_kobo_shelves=False): def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
new_tags_last_modified = sync_token.tags_last_modified new_tags_last_modified = sync_token.tags_last_modified
# transmit all archived shelfs independent of last sync (why should this matter?)
for shelf in ub.session.query(ub.ShelfArchive).filter( for shelf in ub.session.query(ub.ShelfArchive).filter(ub.ShelfArchive.user_id == current_user.id):
func.datetime(ub.ShelfArchive.last_modified) > sync_token.tags_last_modified,
ub.ShelfArchive.user_id == current_user.id
):
new_tags_last_modified = max(shelf.last_modified, new_tags_last_modified) new_tags_last_modified = max(shelf.last_modified, new_tags_last_modified)
sync_results.append({ sync_results.append({
"DeletedTag": { "DeletedTag": {
@ -684,7 +692,6 @@ def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
ub.session.delete(shelf) ub.session.delete(shelf)
ub.session_commit() ub.session_commit()
extra_filters = [] extra_filters = []
if only_kobo_shelves: if only_kobo_shelves:
for shelf in ub.session.query(ub.Shelf).filter( for shelf in ub.session.query(ub.Shelf).filter(
@ -717,7 +724,6 @@ def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
*extra_filters *extra_filters
).distinct().order_by(func.datetime(ub.Shelf.last_modified).asc()) ).distinct().order_by(func.datetime(ub.Shelf.last_modified).asc())
for shelf in shelflist: for shelf in shelflist:
if not shelf_lib.check_shelf_view_permissions(shelf): if not shelf_lib.check_shelf_view_permissions(shelf):
continue continue
@ -753,7 +759,7 @@ def create_kobo_tag(shelf):
for book_shelf in shelf.books: for book_shelf in shelf.books:
book = calibre_db.get_book(book_shelf.book_id) book = calibre_db.get_book(book_shelf.book_id)
if not book: if not book:
log.info(u"Book (id: %s) in BookShelf (id: %s) not found in book database", book_shelf.book_id, shelf.id) log.info("Book (id: %s) in BookShelf (id: %s) not found in book database", book_shelf.book_id, shelf.id)
continue continue
tag["Items"].append( tag["Items"].append(
{ {
@ -763,13 +769,14 @@ def create_kobo_tag(shelf):
) )
return {"Tag": tag} return {"Tag": tag}
@csrf.exempt @csrf.exempt
@kobo.route("/v1/library/<book_uuid>/state", methods=["GET", "PUT"]) @kobo.route("/v1/library/<book_uuid>/state", methods=["GET", "PUT"])
@requires_kobo_auth @requires_kobo_auth
def HandleStateRequest(book_uuid): def HandleStateRequest(book_uuid):
book = calibre_db.get_book_by_uuid(book_uuid) book = calibre_db.get_book_by_uuid(book_uuid)
if not book or not book.data: if not book or not book.data:
log.info(u"Book %s not found in database", book_uuid) log.info("Book %s not found in database", book_uuid)
return redirect_or_proxy_request() return redirect_or_proxy_request()
kobo_reading_state = get_or_create_reading_state(book.id) kobo_reading_state = get_or_create_reading_state(book.id)
@ -807,7 +814,7 @@ def HandleStateRequest(book_uuid):
book_read = kobo_reading_state.book_read_link book_read = kobo_reading_state.book_read_link
new_book_read_status = get_ub_read_status(request_status_info["Status"]) new_book_read_status = get_ub_read_status(request_status_info["Status"])
if new_book_read_status == ub.ReadBook.STATUS_IN_PROGRESS \ if new_book_read_status == ub.ReadBook.STATUS_IN_PROGRESS \
and new_book_read_status != book_read.read_status: and new_book_read_status != book_read.read_status:
book_read.times_started_reading += 1 book_read.times_started_reading += 1
book_read.last_time_started_reading = datetime.datetime.utcnow() book_read.last_time_started_reading = datetime.datetime.utcnow()
book_read.read_status = new_book_read_status book_read.read_status = new_book_read_status
@ -847,7 +854,7 @@ def get_ub_read_status(kobo_read_status):
def get_or_create_reading_state(book_id): def get_or_create_reading_state(book_id):
book_read = ub.session.query(ub.ReadBook).filter(ub.ReadBook.book_id == book_id, book_read = ub.session.query(ub.ReadBook).filter(ub.ReadBook.book_id == book_id,
ub.ReadBook.user_id == current_user.id).one_or_none() ub.ReadBook.user_id == int(current_user.id)).one_or_none()
if not book_read: if not book_read:
book_read = ub.ReadBook(user_id=current_user.id, book_id=book_id) book_read = ub.ReadBook(user_id=current_user.id, book_id=book_id)
if not book_read.kobo_reading_state: if not book_read.kobo_reading_state:
@ -911,13 +918,12 @@ def get_current_bookmark_response(current_bookmark):
} }
return resp return resp
@kobo.route("/<book_uuid>/<width>/<height>/<isGreyscale>/image.jpg", defaults={'Quality': ""}) @kobo.route("/<book_uuid>/<width>/<height>/<isGreyscale>/image.jpg", defaults={'Quality': ""})
@kobo.route("/<book_uuid>/<width>/<height>/<Quality>/<isGreyscale>/image.jpg") @kobo.route("/<book_uuid>/<width>/<height>/<Quality>/<isGreyscale>/image.jpg")
@requires_kobo_auth @requires_kobo_auth
def HandleCoverImageRequest(book_uuid, width, height,Quality, isGreyscale): def HandleCoverImageRequest(book_uuid, width, height, Quality, isGreyscale):
book_cover = helper.get_book_cover_with_uuid( book_cover = helper.get_book_cover_with_uuid(book_uuid, resolution=COVER_THUMBNAIL_SMALL)
book_uuid, use_generic_cover_on_failure=False
)
if not book_cover: if not book_cover:
if config.config_kobo_proxy: if config.config_kobo_proxy:
log.debug("Cover for unknown book: %s proxied to kobo" % book_uuid) log.debug("Cover for unknown book: %s proxied to kobo" % book_uuid)
@ -942,25 +948,16 @@ def TopLevelEndpoint():
@kobo.route("/v1/library/<book_uuid>", methods=["DELETE"]) @kobo.route("/v1/library/<book_uuid>", methods=["DELETE"])
@requires_kobo_auth @requires_kobo_auth
def HandleBookDeletionRequest(book_uuid): def HandleBookDeletionRequest(book_uuid):
log.info("Kobo book deletion request received for book %s" % book_uuid) log.info("Kobo book delete request received for book %s" % book_uuid)
book = calibre_db.get_book_by_uuid(book_uuid) book = calibre_db.get_book_by_uuid(book_uuid)
if not book: if not book:
log.info(u"Book %s not found in database", book_uuid) log.info("Book %s not found in database", book_uuid)
return redirect_or_proxy_request() return redirect_or_proxy_request()
book_id = book.id book_id = book.id
archived_book = ( is_archived = kobo_sync_status.change_archived_books(book_id, True)
ub.session.query(ub.ArchivedBook) if is_archived:
.filter(ub.ArchivedBook.book_id == book_id) kobo_sync_status.remove_synced_book(book_id)
.first()
)
if not archived_book:
archived_book = ub.ArchivedBook(user_id=current_user.id, book_id=book_id)
archived_book.is_archived = True
archived_book.last_modified = datetime.datetime.utcnow()
ub.session.merge(archived_book)
ub.session_commit()
return "", 204 return "", 204
@ -968,7 +965,7 @@ def HandleBookDeletionRequest(book_uuid):
@csrf.exempt @csrf.exempt
@kobo.route("/v1/library/<dummy>", methods=["DELETE", "GET"]) @kobo.route("/v1/library/<dummy>", methods=["DELETE", "GET"])
def HandleUnimplementedRequest(dummy=None): def HandleUnimplementedRequest(dummy=None):
log.debug("Unimplemented Library Request received: %s", request.base_url) log.debug("Unimplemented Library Request received: %s (request is forwarded to kobo if configured)", request.base_url)
return redirect_or_proxy_request() return redirect_or_proxy_request()
@ -980,22 +977,47 @@ def HandleUnimplementedRequest(dummy=None):
@kobo.route("/v1/user/recommendations", methods=["GET", "POST"]) @kobo.route("/v1/user/recommendations", methods=["GET", "POST"])
@kobo.route("/v1/analytics/<dummy>", methods=["GET", "POST"]) @kobo.route("/v1/analytics/<dummy>", methods=["GET", "POST"])
def HandleUserRequest(dummy=None): def HandleUserRequest(dummy=None):
log.debug("Unimplemented User Request received: %s", request.base_url) log.debug("Unimplemented User Request received: %s (request is forwarded to kobo if configured)", request.base_url)
return redirect_or_proxy_request() return redirect_or_proxy_request()
@csrf.exempt
@kobo.route("/v1/user/loyalty/benefits", methods=["GET"])
def handle_benefits():
if config.config_kobo_proxy:
return redirect_or_proxy_request()
else:
return make_response(jsonify({"Benefits": {}}))
@csrf.exempt
@kobo.route("/v1/analytics/gettests", methods=["GET", "POST"])
def handle_getests():
if config.config_kobo_proxy:
return redirect_or_proxy_request()
else:
testkey = request.headers.get("X-Kobo-userkey", "")
return make_response(jsonify({"Result": "Success", "TestKey": testkey, "Tests": {}}))
@csrf.exempt @csrf.exempt
@kobo.route("/v1/products/<dummy>/prices", methods=["GET", "POST"]) @kobo.route("/v1/products/<dummy>/prices", methods=["GET", "POST"])
@kobo.route("/v1/products/<dummy>/recommendations", methods=["GET", "POST"]) @kobo.route("/v1/products/<dummy>/recommendations", methods=["GET", "POST"])
@kobo.route("/v1/products/<dummy>/nextread", methods=["GET", "POST"]) @kobo.route("/v1/products/<dummy>/nextread", methods=["GET", "POST"])
@kobo.route("/v1/products/<dummy>/reviews", methods=["GET", "POST"]) @kobo.route("/v1/products/<dummy>/reviews", methods=["GET", "POST"])
@kobo.route("/v1/products/featured/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/featured/", methods=["GET", "POST"])
@kobo.route("/v1/products/books/external/<dummy>", methods=["GET", "POST"]) @kobo.route("/v1/products/books/external/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/books/series/<dummy>", methods=["GET", "POST"]) @kobo.route("/v1/products/books/series/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/books/<dummy>", methods=["GET", "POST"]) @kobo.route("/v1/products/books/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/books/<dummy>/", methods=["GET", "POST"])
@kobo.route("/v1/products/dailydeal", methods=["GET", "POST"]) @kobo.route("/v1/products/dailydeal", methods=["GET", "POST"])
@kobo.route("/v1/products/deals", methods=["GET", "POST"])
@kobo.route("/v1/products", methods=["GET", "POST"]) @kobo.route("/v1/products", methods=["GET", "POST"])
@kobo.route("/v1/affiliate", methods=["GET", "POST"])
@kobo.route("/v1/deals", methods=["GET", "POST"])
def HandleProductsRequest(dummy=None): def HandleProductsRequest(dummy=None):
log.debug("Unimplemented Products Request received: %s", request.base_url) log.debug("Unimplemented Products Request received: %s (request is forwarded to kobo if configured)", request.base_url)
return redirect_or_proxy_request() return redirect_or_proxy_request()
@ -1005,7 +1027,7 @@ def make_calibre_web_auth_response():
content = request.get_json() content = request.get_json()
AccessToken = base64.b64encode(os.urandom(24)).decode('utf-8') AccessToken = base64.b64encode(os.urandom(24)).decode('utf-8')
RefreshToken = base64.b64encode(os.urandom(24)).decode('utf-8') RefreshToken = base64.b64encode(os.urandom(24)).decode('utf-8')
return make_response( return make_response(
jsonify( jsonify(
{ {
"AccessToken": AccessToken, "AccessToken": AccessToken,
@ -1143,14 +1165,16 @@ def NATIVE_KOBO_RESOURCES():
"eula_page": "https://www.kobo.com/termsofuse?style=onestore", "eula_page": "https://www.kobo.com/termsofuse?style=onestore",
"exchange_auth": "https://storeapi.kobo.com/v1/auth/exchange", "exchange_auth": "https://storeapi.kobo.com/v1/auth/exchange",
"external_book": "https://storeapi.kobo.com/v1/products/books/external/{Ids}", "external_book": "https://storeapi.kobo.com/v1/products/books/external/{Ids}",
"facebook_sso_page": "https://authorize.kobo.com/signin/provider/Facebook/login?returnUrl=http://store.kobobooks.com/", "facebook_sso_page":
"https://authorize.kobo.com/signin/provider/Facebook/login?returnUrl=http://store.kobobooks.com/",
"featured_list": "https://storeapi.kobo.com/v1/products/featured/{FeaturedListId}", "featured_list": "https://storeapi.kobo.com/v1/products/featured/{FeaturedListId}",
"featured_lists": "https://storeapi.kobo.com/v1/products/featured", "featured_lists": "https://storeapi.kobo.com/v1/products/featured",
"free_books_page": { "free_books_page": {
"EN": "https://www.kobo.com/{region}/{language}/p/free-ebooks", "EN": "https://www.kobo.com/{region}/{language}/p/free-ebooks",
"FR": "https://www.kobo.com/{region}/{language}/p/livres-gratuits", "FR": "https://www.kobo.com/{region}/{language}/p/livres-gratuits",
"IT": "https://www.kobo.com/{region}/{language}/p/libri-gratuiti", "IT": "https://www.kobo.com/{region}/{language}/p/libri-gratuiti",
"NL": "https://www.kobo.com/{region}/{language}/List/bekijk-het-overzicht-van-gratis-ebooks/QpkkVWnUw8sxmgjSlCbJRg", "NL": "https://www.kobo.com/{region}/{language}/"
"List/bekijk-het-overzicht-van-gratis-ebooks/QpkkVWnUw8sxmgjSlCbJRg",
"PT": "https://www.kobo.com/{region}/{language}/p/livros-gratis", "PT": "https://www.kobo.com/{region}/{language}/p/livros-gratis",
}, },
"fte_feedback": "https://storeapi.kobo.com/v1/products/ftefeedback", "fte_feedback": "https://storeapi.kobo.com/v1/products/ftefeedback",
@ -1175,7 +1199,8 @@ def NATIVE_KOBO_RESOURCES():
"library_stack": "https://storeapi.kobo.com/v1/user/library/stacks/{LibraryItemId}", "library_stack": "https://storeapi.kobo.com/v1/user/library/stacks/{LibraryItemId}",
"library_sync": "https://storeapi.kobo.com/v1/library/sync", "library_sync": "https://storeapi.kobo.com/v1/library/sync",
"love_dashboard_page": "https://store.kobobooks.com/{culture}/kobosuperpoints", "love_dashboard_page": "https://store.kobobooks.com/{culture}/kobosuperpoints",
"love_points_redemption_page": "https://store.kobobooks.com/{culture}/KoboSuperPointsRedemption?productId={ProductId}", "love_points_redemption_page":
"https://store.kobobooks.com/{culture}/KoboSuperPointsRedemption?productId={ProductId}",
"magazine_landing_page": "https://store.kobobooks.com/emagazines", "magazine_landing_page": "https://store.kobobooks.com/emagazines",
"notifications_registration_issue": "https://storeapi.kobo.com/v1/notifications/registration", "notifications_registration_issue": "https://storeapi.kobo.com/v1/notifications/registration",
"oauth_host": "https://oauth.kobo.com", "oauth_host": "https://oauth.kobo.com",
@ -1191,7 +1216,8 @@ def NATIVE_KOBO_RESOURCES():
"product_recommendations": "https://storeapi.kobo.com/v1/products/{ProductId}/recommendations", "product_recommendations": "https://storeapi.kobo.com/v1/products/{ProductId}/recommendations",
"product_reviews": "https://storeapi.kobo.com/v1/products/{ProductIds}/reviews", "product_reviews": "https://storeapi.kobo.com/v1/products/{ProductIds}/reviews",
"products": "https://storeapi.kobo.com/v1/products", "products": "https://storeapi.kobo.com/v1/products",
"provider_external_sign_in_page": "https://authorize.kobo.com/ExternalSignIn/{providerName}?returnUrl=http://store.kobobooks.com/", "provider_external_sign_in_page":
"https://authorize.kobo.com/ExternalSignIn/{providerName}?returnUrl=http://store.kobobooks.com/",
"purchase_buy": "https://www.kobo.com/checkout/createpurchase/", "purchase_buy": "https://www.kobo.com/checkout/createpurchase/",
"purchase_buy_templated": "https://www.kobo.com/{culture}/checkout/createpurchase/{ProductId}", "purchase_buy_templated": "https://www.kobo.com/{culture}/checkout/createpurchase/{ProductId}",
"quickbuy_checkout": "https://storeapi.kobo.com/v1/store/quickbuy/{PurchaseId}/checkout", "quickbuy_checkout": "https://storeapi.kobo.com/v1/store/quickbuy/{PurchaseId}/checkout",

132
cps/kobo_auth.py
View File

@ -62,6 +62,7 @@ particular calls to non-Kobo specific endpoints such as the CalibreWeb book down
from binascii import hexlify from binascii import hexlify
from datetime import datetime from datetime import datetime
from os import urandom from os import urandom
from functools import wraps
from flask import g, Blueprint, url_for, abort, request from flask import g, Blueprint, url_for, abort, request
from flask_login import login_user, current_user, login_required from flask_login import login_user, current_user, login_required
@ -70,20 +71,61 @@ from flask_babel import gettext as _
from . import logger, config, calibre_db, db, helper, ub, lm from . import logger, config, calibre_db, db, helper, ub, lm
from .render_template import render_title_template from .render_template import render_title_template
try:
from functools import wraps
except ImportError:
pass # We're not using Python 3
log = logger.create() log = logger.create()
kobo_auth = Blueprint("kobo_auth", __name__, url_prefix="/kobo_auth")
def register_url_value_preprocessor(kobo):
@kobo.url_value_preprocessor @kobo_auth.route("/generate_auth_token/<int:user_id>")
# pylint: disable=unused-variable @login_required
def pop_auth_token(__, values): def generate_auth_token(user_id):
g.auth_token = values.pop("auth_token") warning = False
host_list = request.host.rsplit(':')
if len(host_list) == 1:
host = ':'.join(host_list)
else:
host = ':'.join(host_list[0:-1])
if host.startswith('127.') or host.lower() == 'localhost' or host.startswith('[::ffff:7f') or host == "[::1]":
warning = _('Please access Calibre-Web from non localhost to get valid api_endpoint for kobo device')
# Generate auth token if none is existing for this user
auth_token = ub.session.query(ub.RemoteAuthToken).filter(
ub.RemoteAuthToken.user_id == user_id
).filter(ub.RemoteAuthToken.token_type==1).first()
if not auth_token:
auth_token = ub.RemoteAuthToken()
auth_token.user_id = user_id
auth_token.expiration = datetime.max
auth_token.auth_token = (hexlify(urandom(16))).decode("utf-8")
auth_token.token_type = 1
ub.session.add(auth_token)
ub.session_commit()
books = calibre_db.session.query(db.Books).join(db.Data).all()
for book in books:
formats = [data.format for data in book.data]
if 'KEPUB' not in formats and config.config_kepubifypath and 'EPUB' in formats:
helper.convert_book_format(book.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
return render_title_template(
"generate_kobo_auth_url.html",
title=_("Kobo Setup"),
auth_token=auth_token.auth_token,
warning = warning
)
@kobo_auth.route("/deleteauthtoken/<int:user_id>", methods=["POST"])
@login_required
def delete_auth_token(user_id):
# Invalidate any previously generated Kobo Auth token for this user
ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.user_id == user_id)\
.filter(ub.RemoteAuthToken.token_type==1).delete()
return ub.session_commit()
def disable_failed_auth_redirect_for_blueprint(bp): def disable_failed_auth_redirect_for_blueprint(bp):
@ -97,6 +139,13 @@ def get_auth_token():
return None return None
def register_url_value_preprocessor(kobo):
@kobo.url_value_preprocessor
# pylint: disable=unused-variable
def pop_auth_token(__, values):
g.auth_token = values.pop("auth_token")
def requires_kobo_auth(f): def requires_kobo_auth(f):
@wraps(f) @wraps(f)
def inner(*args, **kwargs): def inner(*args, **kwargs):
@ -114,64 +163,3 @@ def requires_kobo_auth(f):
log.debug("Received Kobo request without a recognizable auth token.") log.debug("Received Kobo request without a recognizable auth token.")
return abort(401) return abort(401)
return inner return inner
kobo_auth = Blueprint("kobo_auth", __name__, url_prefix="/kobo_auth")
@kobo_auth.route("/generate_auth_token/<int:user_id>")
@login_required
def generate_auth_token(user_id):
host_list = request.host.rsplit(':')
if len(host_list) == 1:
host = ':'.join(host_list)
else:
host = ':'.join(host_list[0:-1])
if host.startswith('127.') or host.lower() == 'localhost' or host.startswith('[::ffff:7f'):
warning = _('PLease access calibre-web from non localhost to get valid api_endpoint for kobo device')
return render_title_template(
"generate_kobo_auth_url.html",
title=_(u"Kobo Setup"),
warning = warning
)
else:
# Invalidate any prevously generated Kobo Auth token for this user.
auth_token = ub.session.query(ub.RemoteAuthToken).filter(
ub.RemoteAuthToken.user_id == user_id
).filter(ub.RemoteAuthToken.token_type==1).first()
if not auth_token:
auth_token = ub.RemoteAuthToken()
auth_token.user_id = user_id
auth_token.expiration = datetime.max
auth_token.auth_token = (hexlify(urandom(16))).decode("utf-8")
auth_token.token_type = 1
ub.session.add(auth_token)
ub.session_commit()
books = calibre_db.session.query(db.Books).join(db.Data).all()
for book in books:
formats = [data.format for data in book.data]
if not 'KEPUB' in formats and config.config_kepubifypath and 'EPUB' in formats:
helper.convert_book_format(book.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
return render_title_template(
"generate_kobo_auth_url.html",
title=_(u"Kobo Setup"),
kobo_auth_url=url_for(
"kobo.TopLevelEndpoint", auth_token=auth_token.auth_token, _external=True
),
warning = False
)
@kobo_auth.route("/deleteauthtoken/<int:user_id>")
@login_required
def delete_auth_token(user_id):
# Invalidate any prevously generated Kobo Auth token for this user.
ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.user_id == user_id)\
.filter(ub.RemoteAuthToken.token_type==1).delete()
return ub.session_commit()

91
cps/kobo_sync_status.py
View File

@ -20,52 +20,69 @@
from flask_login import current_user from flask_login import current_user
from . import ub from . import ub
import datetime import datetime
from sqlalchemy.sql.expression import or_ from sqlalchemy.sql.expression import or_, and_, true
from sqlalchemy import exc
# Add the current book id to kobo_synced_books table for current user, if entry is already present,
# do nothing (safety precaution)
def add_synced_books(book_id): def add_synced_books(book_id):
synced_book = ub.KoboSyncedBooks() is_present = ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id)\
synced_book.user_id = current_user.id .filter(ub.KoboSyncedBooks.user_id == current_user.id).count()
synced_book.book_id = book_id if not is_present:
ub.session.add(synced_book) synced_book = ub.KoboSyncedBooks()
ub.session_commit() synced_book.user_id = current_user.id
synced_book.book_id = book_id
ub.session.add(synced_book)
ub.session_commit()
def remove_synced_book(book_id): # Select all entries of current book in kobo_synced_books table, which are from current user and delete them
ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id).delete() def remove_synced_book(book_id, all=False, session=None):
ub.session_commit() if not all:
user = ub.KoboSyncedBooks.user_id == current_user.id
else:
user = true()
if not session:
ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id).filter(user).delete()
ub.session_commit()
else:
session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id).filter(user).delete()
ub.session_commit(_session=session)
def add_archived_books(book_id):
archived_book = (
ub.session.query(ub.ArchivedBook) def change_archived_books(book_id, state=None, message=None):
.filter(ub.ArchivedBook.book_id == book_id) archived_book = ub.session.query(ub.ArchivedBook).filter(and_(ub.ArchivedBook.user_id == int(current_user.id),
.first() ub.ArchivedBook.book_id == book_id)).first()
)
if not archived_book: if not archived_book:
archived_book = ub.ArchivedBook(user_id=current_user.id, book_id=book_id) archived_book = ub.ArchivedBook(user_id=current_user.id, book_id=book_id)
archived_book.is_archived = True
archived_book.last_modified = datetime.datetime.utcnow() archived_book.is_archived = state if state else not archived_book.is_archived
archived_book.last_modified = datetime.datetime.utcnow() # toDo. Check utc timestamp
ub.session.merge(archived_book) ub.session.merge(archived_book)
ub.session_commit() ub.session_commit(message)
return archived_book.is_archived
# select all books which are synced by the current user and do not belong to a synced shelf and set them to archive
# select all shelves from current user which are synced and do not belong to the "only sync" shelves
def update_on_sync_shelfs(user_id):
books_to_archive = (ub.session.query(ub.KoboSyncedBooks)
.join(ub.BookShelf, ub.KoboSyncedBooks.book_id == ub.BookShelf.book_id, isouter=True)
.join(ub.Shelf, ub.Shelf.user_id == user_id, isouter=True)
.filter(or_(ub.Shelf.kobo_sync == 0, ub.Shelf.kobo_sync == None))
.filter(ub.KoboSyncedBooks.user_id == user_id).all())
for b in books_to_archive:
change_archived_books(b.book_id, True)
ub.session.query(ub.KoboSyncedBooks) \
.filter(ub.KoboSyncedBooks.book_id == b.book_id) \
.filter(ub.KoboSyncedBooks.user_id == user_id).delete()
ub.session_commit()
# select all books which are synced by the current user and do not belong to a synced shelf and them to archive # Search all shelf which are currently not synced
# select all shelfs from current user which are synced and do not belong to the "only sync" shelfs shelves_to_archive = ub.session.query(ub.Shelf).filter(ub.Shelf.user_id == user_id).filter(
def update_on_sync_shelfs(content_id): ub.Shelf.kobo_sync == 0).all()
books_to_archive = (ub.session.query(ub.KoboSyncedBooks) for a in shelves_to_archive:
.join(ub.BookShelf, ub.KoboSyncedBooks.book_id == ub.BookShelf.book_id, isouter=True) ub.session.add(ub.ShelfArchive(uuid=a.uuid, user_id=user_id))
.join(ub.Shelf, ub.Shelf.user_id == content_id, isouter=True) ub.session_commit()
.filter(or_(ub.Shelf.kobo_sync == 0, ub.Shelf.kobo_sync == None))
.filter(ub.KoboSyncedBooks.user_id == content_id).all())
for b in books_to_archive:
add_archived_books(b.book_id)
ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == b.book_id).filter(ub.KoboSyncedBooks.user_id == content_id).delete()
ub.session_commit()
shelfs_to_archive = ub.session.query(ub.Shelf).filter(ub.Shelf.user_id == content_id).filter(
ub.Shelf.kobo_sync == 0).all()
for a in shelfs_to_archive:
ub.session.add(ub.ShelfArchive(uuid=a.uuid, user_id=content_id))
ub.session_commit()

31
cps/logger.py
View File

@ -42,23 +42,17 @@ logging.addLevelName(logging.CRITICAL, "CRIT")
class _Logger(logging.Logger): class _Logger(logging.Logger):
def debug_or_exception(self, message, *args, **kwargs): def error_or_exception(self, message, stacklevel=2, *args, **kwargs):
if sys.version_info > (3, 7): if sys.version_info > (3, 7):
if is_debug_enabled(): if is_debug_enabled():
self.exception(message, stacklevel=2, *args, **kwargs) self.exception(message, stacklevel=stacklevel, *args, **kwargs)
else: else:
self.error(message, stacklevel=2, *args, **kwargs) self.error(message, stacklevel=stacklevel, *args, **kwargs)
elif sys.version_info > (3, 0): else:
if is_debug_enabled(): if is_debug_enabled():
self.exception(message, stack_info=True, *args, **kwargs) self.exception(message, stack_info=True, *args, **kwargs)
else: else:
self.error(message, *args, **kwargs) self.error(message, *args, **kwargs)
else:
if is_debug_enabled():
self.exception(message, *args, **kwargs)
else:
self.error(message, *args, **kwargs)
def debug_no_auth(self, message, *args, **kwargs): def debug_no_auth(self, message, *args, **kwargs):
message = message.strip("\r\n") message = message.strip("\r\n")
@ -71,6 +65,7 @@ class _Logger(logging.Logger):
def get(name=None): def get(name=None):
return logging.getLogger(name) return logging.getLogger(name)
def create(): def create():
parent_frame = inspect.stack(0)[1] parent_frame = inspect.stack(0)[1]
if hasattr(parent_frame, 'frame'): if hasattr(parent_frame, 'frame'):
@ -80,9 +75,11 @@ def create():
parent_module = inspect.getmodule(parent_frame) parent_module = inspect.getmodule(parent_frame)
return get(parent_module.__name__) return get(parent_module.__name__)
def is_debug_enabled(): def is_debug_enabled():
return logging.root.level <= logging.DEBUG return logging.root.level <= logging.DEBUG
def is_info_enabled(logger): def is_info_enabled(logger):
return logging.getLogger(logger).level <= logging.INFO return logging.getLogger(logger).level <= logging.INFO
@ -119,10 +116,10 @@ def get_accesslogfile(log_file):
def setup(log_file, log_level=None): def setup(log_file, log_level=None):
''' """
Configure the logging output. Configure the logging output.
May be called multiple times. May be called multiple times.
''' """
log_level = log_level or DEFAULT_LOG_LEVEL log_level = log_level or DEFAULT_LOG_LEVEL
logging.setLoggerClass(_Logger) logging.setLoggerClass(_Logger)
logging.getLogger(__package__).setLevel(log_level) logging.getLogger(__package__).setLevel(log_level)
@ -132,7 +129,7 @@ def setup(log_file, log_level=None):
# avoid spamming the log with debug messages from libraries # avoid spamming the log with debug messages from libraries
r.setLevel(log_level) r.setLevel(log_level)
# Otherwise name get's destroyed on windows # Otherwise, name gets destroyed on Windows
if log_file != LOG_TO_STDERR and log_file != LOG_TO_STDOUT: if log_file != LOG_TO_STDERR and log_file != LOG_TO_STDOUT:
log_file = _absolute_log_file(log_file, DEFAULT_LOG_FILE) log_file = _absolute_log_file(log_file, DEFAULT_LOG_FILE)
@ -164,13 +161,14 @@ def setup(log_file, log_level=None):
r.removeHandler(h) r.removeHandler(h)
h.close() h.close()
r.addHandler(file_handler) r.addHandler(file_handler)
logging.captureWarnings(True)
return "" if log_file == DEFAULT_LOG_FILE else log_file return "" if log_file == DEFAULT_LOG_FILE else log_file
def create_access_log(log_file, log_name, formatter): def create_access_log(log_file, log_name, formatter):
''' """
One-time configuration for the web server's access log. One-time configuration for the web server's access log.
''' """
log_file = _absolute_log_file(log_file, DEFAULT_ACCESS_LOG) log_file = _absolute_log_file(log_file, DEFAULT_ACCESS_LOG)
logging.debug("access log: %s", log_file) logging.debug("access log: %s", log_file)
@ -187,8 +185,7 @@ def create_access_log(log_file, log_name, formatter):
file_handler.setFormatter(formatter) file_handler.setFormatter(formatter)
access_log.addHandler(file_handler) access_log.addHandler(file_handler)
return access_log, \ return access_log, "" if _absolute_log_file(log_file, DEFAULT_ACCESS_LOG) == DEFAULT_ACCESS_LOG else log_file
"" if _absolute_log_file(log_file, DEFAULT_ACCESS_LOG) == DEFAULT_ACCESS_LOG else log_file
# Enable logging of smtp lib debug output # Enable logging of smtp lib debug output

73
cps/main.py Normal file
View File

@ -0,0 +1,73 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2012-2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
from . import create_app
from .jinjia import jinjia
from .remotelogin import remotelogin
def main():
app = create_app()
from .web import web
from .opds import opds
from .admin import admi
from .gdrive import gdrive
from .editbooks import editbook
from .about import about
from .search import search
from .search_metadata import meta
from .shelf import shelf
from .tasks_status import tasks
from .error_handler import init_errorhandler
try:
from .kobo import kobo, get_kobo_activated
from .kobo_auth import kobo_auth
kobo_available = get_kobo_activated()
except (ImportError, AttributeError): # Catch also error for not installed flask-WTF (missing csrf decorator)
kobo_available = False
try:
from .oauth_bb import oauth
oauth_available = True
except ImportError:
oauth_available = False
from . import web_server
init_errorhandler()
app.register_blueprint(search)
app.register_blueprint(tasks)
app.register_blueprint(web)
app.register_blueprint(opds)
app.register_blueprint(jinjia)
app.register_blueprint(about)
app.register_blueprint(shelf)
app.register_blueprint(admi)
app.register_blueprint(remotelogin)
app.register_blueprint(meta)
app.register_blueprint(gdrive)
app.register_blueprint(editbook)
if kobo_available:
app.register_blueprint(kobo)
app.register_blueprint(kobo_auth)
if oauth_available:
app.register_blueprint(oauth)
success = web_server.start()
sys.exit(0 if success else 1)

141
cps/metadata_provider/amazon.py Normal file
View File

@ -0,0 +1,141 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 quarz12
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import concurrent.futures
import requests
from bs4 import BeautifulSoup as BS # requirement
from typing import List, Optional
try:
import cchardet #optional for better speed
except ImportError:
pass
from cps import logger
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
import cps.logger as logger
#from time import time
from operator import itemgetter
log = logger.create()
log = logger.create()
class Amazon(Metadata):
__name__ = "Amazon"
__id__ = "amazon"
headers = {'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'sec-gpc': '1',
'sec-fetch-site': 'none',
'sec-fetch-mode': 'navigate',
'sec-fetch-user': '?1',
'sec-fetch-dest': 'document',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9'}
session = requests.Session()
session.headers=headers
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
#timer=time()
def inner(link, index) -> [dict, int]:
with self.session as session:
try:
r = session.get(f"https://www.amazon.com/{link}")
r.raise_for_status()
except Exception as ex:
log.warning(ex)
return None
long_soup = BS(r.text, "lxml") #~4sec :/
soup2 = long_soup.find("div", attrs={"cel_widget_id": "dpx-books-ppd_csm_instrumentation_wrapper"})
if soup2 is None:
return None
try:
match = MetaRecord(
title = "",
authors = "",
source=MetaSourceInfo(
id=self.__id__,
description="Amazon Books",
link="https://amazon.com/"
),
url = f"https://www.amazon.com{link}",
#the more searches the slower, these are too hard to find in reasonable time or might not even exist
publisher= "", # very unreliable
publishedDate= "", # very unreliable
id = None, # ?
tags = [] # dont exist on amazon
)
try:
match.description = "\n".join(
soup2.find("div", attrs={"data-feature-name": "bookDescription"}).stripped_strings)\
.replace("\xa0"," ")[:-9].strip().strip("\n")
except (AttributeError, TypeError):
return None # if there is no description it is not a book and therefore should be ignored
try:
match.title = soup2.find("span", attrs={"id": "productTitle"}).text
except (AttributeError, TypeError):
match.title = ""
try:
match.authors = [next(
filter(lambda i: i != " " and i != "\n" and not i.startswith("{"),
x.findAll(text=True))).strip()
for x in soup2.findAll("span", attrs={"class": "author"})]
except (AttributeError, TypeError, StopIteration):
match.authors = ""
try:
match.rating = int(
soup2.find("span", class_="a-icon-alt").text.split(" ")[0].split(".")[
0]) # first number in string
except (AttributeError, ValueError):
match.rating = 0
try:
match.cover = soup2.find("img", attrs={"class": "a-dynamic-image frontImage"})["src"]
except (AttributeError, TypeError):
match.cover = ""
return match, index
except Exception as e:
log.error_or_exception(e)
return None
val = list()
if self.active:
try:
results = self.session.get(
f"https://www.amazon.com/s?k={query.replace(' ', '+')}&i=digital-text&sprefix={query.replace(' ', '+')}"
f"%2Cdigital-text&ref=nb_sb_noss",
headers=self.headers)
results.raise_for_status()
except requests.exceptions.HTTPError as e:
log.error_or_exception(e)
return []
except Exception as e:
log.warning(e)
return []
soup = BS(results.text, 'html.parser')
links_list = [next(filter(lambda i: "digital-text" in i["href"], x.findAll("a")))["href"] for x in
soup.findAll("div", attrs={"data-component-type": "s-search-result"})]
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
fut = {executor.submit(inner, link, index) for index, link in enumerate(links_list[:5])}
val = list(map(lambda x : x.result() ,concurrent.futures.as_completed(fut)))
result = list(filter(lambda x: x, val))
return [x[0] for x in sorted(result, key=itemgetter(1))] #sort by amazons listing order for best relevance

95
cps/metadata_provider/comicvine.py
View File

@ -17,49 +17,76 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# ComicVine api document: https://comicvine.gamespot.com/api/documentation # ComicVine api document: https://comicvine.gamespot.com/api/documentation
from typing import Dict, List, Optional
from urllib.parse import quote
import requests import requests
from cps.services.Metadata import Metadata from cps import logger
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
class ComicVine(Metadata): class ComicVine(Metadata):
__name__ = "ComicVine" __name__ = "ComicVine"
__id__ = "comicvine" __id__ = "comicvine"
DESCRIPTION = "ComicVine Books"
META_URL = "https://comicvine.gamespot.com/"
API_KEY = "57558043c53943d5d1e96a9ad425b0eb85532ee6"
BASE_URL = (
f"https://comicvine.gamespot.com/api/search?api_key={API_KEY}"
f"&resources=issue&query="
)
QUERY_PARAMS = "&sort=name:desc&format=json"
HEADERS = {"User-Agent": "Not Evil Browser"}
def search(self, query, __): def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
val = list() val = list()
apikey = "57558043c53943d5d1e96a9ad425b0eb85532ee6"
if self.active: if self.active:
headers = { title_tokens = list(self.get_title_tokens(query, strip_joiners=False))
'User-Agent': 'Not Evil Browser' if title_tokens:
} tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = "%20".join(tokens)
result = requests.get("https://comicvine.gamespot.com/api/search?api_key=" try:
+ apikey + "&resources=issue&query=" + query + "&sort=name:desc&format=json", headers=headers) result = requests.get(
for r in result.json()['results']: f"{ComicVine.BASE_URL}{query}{ComicVine.QUERY_PARAMS}",
seriesTitle = r['volume'].get('name', "") headers=ComicVine.HEADERS,
if r.get('store_date'): )
dateFomers = r.get('store_date') result.raise_for_status()
else: except Exception as e:
dateFomers = r.get('date_added') log.warning(e)
v = dict() return None
v['id'] = r['id'] for result in result.json()["results"]:
v['title'] = seriesTitle + " #" + r.get('issue_number', "0") + " - " + ( r.get('name', "") or "") match = self._parse_search_result(
v['authors'] = r.get('authors', []) result=result, generic_cover=generic_cover, locale=locale
v['description'] = r.get('description', "") )
v['publisher'] = "" val.append(match)
v['publishedDate'] = dateFomers
v['tags'] = ["Comics", seriesTitle]
v['rating'] = 0
v['series'] = seriesTitle
v['cover'] = r['image'].get('original_url')
v['source'] = {
"id": self.__id__,
"description": "ComicVine Books",
"link": "https://comicvine.gamespot.com/"
}
v['url'] = r.get('site_detail_url', "")
val.append(v)
return val return val
def _parse_search_result(
self, result: Dict, generic_cover: str, locale: str
) -> MetaRecord:
series = result["volume"].get("name", "")
series_index = result.get("issue_number", 0)
issue_name = result.get("name", "")
match = MetaRecord(
id=result["id"],
title=f"{series}#{series_index} - {issue_name}",
authors=result.get("authors", []),
url=result.get("site_detail_url", ""),
source=MetaSourceInfo(
id=self.__id__,
description=ComicVine.DESCRIPTION,
link=ComicVine.META_URL,
),
series=series,
)
match.cover = result["image"].get("original_url", generic_cover)
match.description = result.get("description", "")
match.publishedDate = result.get("store_date", result.get("date_added"))
match.series_index = series_index
match.tags = ["Comics", series]
match.identifiers = {"comicvine": match.id}
return match

258
cps/metadata_provider/douban.py Normal file
View File

@ -0,0 +1,258 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 xlivevil
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import re
from concurrent import futures
from typing import List, Optional
import requests
from html2text import HTML2Text
from lxml import etree
from cps import logger
from cps.services.Metadata import Metadata, MetaRecord, MetaSourceInfo
log = logger.create()
def html2text(html: str) -> str:
h2t = HTML2Text()
h2t.body_width = 0
h2t.single_line_break = True
h2t.emphasis_mark = "*"
return h2t.handle(html)
class Douban(Metadata):
__name__ = "豆瓣"
__id__ = "douban"
DESCRIPTION = "豆瓣"
META_URL = "https://book.douban.com/"
SEARCH_JSON_URL = "https://www.douban.com/j/search"
SEARCH_URL = "https://www.douban.com/search"
ID_PATTERN = re.compile(r"sid: (?P<id>\d+),")
AUTHORS_PATTERN = re.compile(r"作者|译者")
PUBLISHER_PATTERN = re.compile(r"出版社")
SUBTITLE_PATTERN = re.compile(r"副标题")
PUBLISHED_DATE_PATTERN = re.compile(r"出版年")
SERIES_PATTERN = re.compile(r"丛书")
IDENTIFIERS_PATTERN = re.compile(r"ISBN|统一书号")
CRITERIA_PATTERN = re.compile("criteria = '(.+)'")
TITTLE_XPATH = "//span[@property='v:itemreviewed']"
COVER_XPATH = "//a[@class='nbg']"
INFO_XPATH = "//*[@id='info']//span[@class='pl']"
TAGS_XPATH = "//a[contains(@class, 'tag')]"
DESCRIPTION_XPATH = "//div[@id='link-report']//div[@class='intro']"
RATING_XPATH = "//div[@class='rating_self clearfix']/strong"
session = requests.Session()
session.headers = {
'user-agent':
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36 Edg/98.0.1108.56',
}
def search(self,
query: str,
generic_cover: str = "",
locale: str = "en") -> List[MetaRecord]:
val = []
if self.active:
log.debug(f"start searching {query} on douban")
if title_tokens := list(
self.get_title_tokens(query, strip_joiners=False)):
query = "+".join(title_tokens)
book_id_list = self._get_book_id_list_from_html(query)
if not book_id_list:
log.debug("No search results in Douban")
return []
with futures.ThreadPoolExecutor(
max_workers=5, thread_name_prefix='douban') as executor:
fut = [
executor.submit(self._parse_single_book, book_id,
generic_cover) for book_id in book_id_list
]
val = [
future.result() for future in futures.as_completed(fut)
if future.result()
]
return val
def _get_book_id_list_from_html(self, query: str) -> List[str]:
try:
r = self.session.get(self.SEARCH_URL,
params={
"cat": 1001,
"q": query
})
r.raise_for_status()
except Exception as e:
log.warning(e)
return []
html = etree.HTML(r.content.decode("utf8"))
result_list = html.xpath(self.COVER_XPATH)
return [
self.ID_PATTERN.search(item.get("onclick")).group("id")
for item in result_list[:10]
if self.ID_PATTERN.search(item.get("onclick"))
]
def _get_book_id_list_from_json(self, query: str) -> List[str]:
try:
r = self.session.get(self.SEARCH_JSON_URL,
params={
"cat": 1001,
"q": query
})
r.raise_for_status()
except Exception as e:
log.warning(e)
return []
results = r.json()
if results["total"] == 0:
return []
return [
self.ID_PATTERN.search(item).group("id")
for item in results["items"][:10] if self.ID_PATTERN.search(item)
]
def _parse_single_book(self,
id: str,
generic_cover: str = "") -> Optional[MetaRecord]:
url = f"https://book.douban.com/subject/{id}/"
log.debug(f"start parsing {url}")
try:
r = self.session.get(url)
r.raise_for_status()
except Exception as e:
log.warning(e)
return None
match = MetaRecord(
id=id,
title="",
authors=[],
url=url,
source=MetaSourceInfo(
id=self.__id__,
description=self.DESCRIPTION,
link=self.META_URL,
),
)
html = etree.HTML(r.content.decode("utf8"))
match.title = html.xpath(self.TITTLE_XPATH)[0].text
match.cover = html.xpath(
self.COVER_XPATH)[0].attrib["href"] or generic_cover
try:
rating_num = float(html.xpath(self.RATING_XPATH)[0].text.strip())
except Exception:
rating_num = 0
match.rating = int(-1 * rating_num // 2 * -1) if rating_num else 0
tag_elements = html.xpath(self.TAGS_XPATH)
if len(tag_elements):
match.tags = [tag_element.text for tag_element in tag_elements]
else:
match.tags = self._get_tags(html.text)
description_element = html.xpath(self.DESCRIPTION_XPATH)
if len(description_element):
match.description = html2text(
etree.tostring(description_element[-1]).decode("utf8"))
info = html.xpath(self.INFO_XPATH)
for element in info:
text = element.text
if self.AUTHORS_PATTERN.search(text):
next_element = element.getnext()
while next_element is not None and next_element.tag != "br":
match.authors.append(next_element.text)
next_element = next_element.getnext()
elif self.PUBLISHER_PATTERN.search(text):
if publisher := element.tail.strip():
match.publisher = publisher
else:
match.publisher = element.getnext().text
elif self.SUBTITLE_PATTERN.search(text):
match.title = f'{match.title}:{element.tail.strip()}'
elif self.PUBLISHED_DATE_PATTERN.search(text):
match.publishedDate = self._clean_date(element.tail.strip())
elif self.SERIES_PATTERN.search(text):
match.series = element.getnext().text
elif i_type := self.IDENTIFIERS_PATTERN.search(text):
match.identifiers[i_type.group()] = element.tail.strip()
return match
def _clean_date(self, date: str) -> str:
"""
Clean up the date string to be in the format YYYY-MM-DD
Examples of possible patterns:
'2014-7-16', '1988年4月', '1995-04', '2021-8', '2020-12-1', '1996年',
'1972', '2004/11/01', '1959年3月北京第1版第1印'
"""
year = date[:4]
moon = "01"
day = "01"
if len(date) > 5:
digit = []
ls = []
for i in range(5, len(date)):
if date[i].isdigit():
digit.append(date[i])
elif digit:
ls.append("".join(digit) if len(digit) ==
2 else f"0{digit[0]}")
digit = []
if digit:
ls.append("".join(digit) if len(digit) ==
2 else f"0{digit[0]}")
moon = ls[0]
if len(ls) > 1:
day = ls[1]
return f"{year}-{moon}-{day}"
def _get_tags(self, text: str) -> List[str]:
tags = []
if criteria := self.CRITERIA_PATTERN.search(text):
tags.extend(
item.replace('7:', '') for item in criteria.group().split('|')
if item.startswith('7:'))
return tags

126
cps/metadata_provider/google.py
View File

@ -17,39 +17,113 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# Google Books api document: https://developers.google.com/books/docs/v1/using # Google Books api document: https://developers.google.com/books/docs/v1/using
from typing import Dict, List, Optional
from urllib.parse import quote
from datetime import datetime
import requests import requests
from cps.services.Metadata import Metadata
from cps import logger
from cps.isoLanguages import get_lang3, get_language_name
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
class Google(Metadata): class Google(Metadata):
__name__ = "Google" __name__ = "Google"
__id__ = "google" __id__ = "google"
DESCRIPTION = "Google Books"
META_URL = "https://books.google.com/"
BOOK_URL = "https://books.google.com/books?id="
SEARCH_URL = "https://www.googleapis.com/books/v1/volumes?q="
ISBN_TYPE = "ISBN_13"
def search(self, query, __): def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
val = list()
if self.active: if self.active:
val = list()
result = requests.get("https://www.googleapis.com/books/v1/volumes?q="+query.replace(" ","+"))
for r in result.json()['items']:
v = dict()
v['id'] = r['id']
v['title'] = r['volumeInfo']['title']
v['authors'] = r['volumeInfo'].get('authors', [])
v['description'] = r['volumeInfo'].get('description', "")
v['publisher'] = r['volumeInfo'].get('publisher', "")
v['publishedDate'] = r['volumeInfo'].get('publishedDate', "")
v['tags'] = r['volumeInfo'].get('categories', [])
v['rating'] = r['volumeInfo'].get('averageRating', 0)
if r['volumeInfo'].get('imageLinks'):
v['cover'] = r['volumeInfo']['imageLinks']['thumbnail'].replace("http://", "https://")
else:
v['cover'] = "/../../../static/generic_cover.jpg"
v['source'] = {
"id": self.__id__,
"description": "Google Books",
"link": "https://books.google.com/"}
v['url'] = "https://books.google.com/books?id=" + r['id']
val.append(v)
return val
title_tokens = list(self.get_title_tokens(query, strip_joiners=False))
if title_tokens:
tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = "+".join(tokens)
try:
results = requests.get(Google.SEARCH_URL + query)
results.raise_for_status()
except Exception as e:
log.warning(e)
return None
for result in results.json().get("items", []):
val.append(
self._parse_search_result(
result=result, generic_cover=generic_cover, locale=locale
)
)
return val
def _parse_search_result(
self, result: Dict, generic_cover: str, locale: str
) -> MetaRecord:
match = MetaRecord(
id=result["id"],
title=result["volumeInfo"]["title"],
authors=result["volumeInfo"].get("authors", []),
url=Google.BOOK_URL + result["id"],
source=MetaSourceInfo(
id=self.__id__,
description=Google.DESCRIPTION,
link=Google.META_URL,
),
)
match.cover = self._parse_cover(result=result, generic_cover=generic_cover)
match.description = result["volumeInfo"].get("description", "")
match.languages = self._parse_languages(result=result, locale=locale)
match.publisher = result["volumeInfo"].get("publisher", "")
try:
datetime.strptime(result["volumeInfo"].get("publishedDate", ""), "%Y-%m-%d")
match.publishedDate = result["volumeInfo"].get("publishedDate", "")
except ValueError:
match.publishedDate = ""
match.rating = result["volumeInfo"].get("averageRating", 0)
match.series, match.series_index = "", 1
match.tags = result["volumeInfo"].get("categories", [])
match.identifiers = {"google": match.id}
match = self._parse_isbn(result=result, match=match)
return match
@staticmethod
def _parse_isbn(result: Dict, match: MetaRecord) -> MetaRecord:
identifiers = result["volumeInfo"].get("industryIdentifiers", [])
for identifier in identifiers:
if identifier.get("type") == Google.ISBN_TYPE:
match.identifiers["isbn"] = identifier.get("identifier")
break
return match
@staticmethod
def _parse_cover(result: Dict, generic_cover: str) -> str:
if result["volumeInfo"].get("imageLinks"):
cover_url = result["volumeInfo"]["imageLinks"]["thumbnail"]
# strip curl in cover
cover_url = cover_url.replace("&edge=curl", "")
# request 800x900 cover image (higher resolution)
cover_url += "&fife=w800-h900"
return cover_url.replace("http://", "https://")
return generic_cover
@staticmethod
def _parse_languages(result: Dict, locale: str) -> List[str]:
language_iso2 = result["volumeInfo"].get("language", "")
languages = (
[get_language_name(locale, get_lang3(language_iso2))]
if language_iso2
else []
)
return languages

350
cps/metadata_provider/lubimyczytac.py Normal file
View File

@ -0,0 +1,350 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2021 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import datetime
import json
import re
from multiprocessing.pool import ThreadPool
from typing import List, Optional, Tuple, Union
from urllib.parse import quote
import requests
from dateutil import parser
from html2text import HTML2Text
from lxml.html import HtmlElement, fromstring, tostring
from markdown2 import Markdown
from cps import logger
from cps.isoLanguages import get_language_name
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
SYMBOLS_TO_TRANSLATE = (
"öÖüÜóÓőŐúÚéÉáÁűŰíÍąĄćĆęĘłŁńŃóÓśŚźŹżŻ",
"oOuUoOoOuUeEaAuUiIaAcCeElLnNoOsSzZzZ",
)
SYMBOL_TRANSLATION_MAP = dict(
[(ord(a), ord(b)) for (a, b) in zip(*SYMBOLS_TO_TRANSLATE)]
)
def get_int_or_float(value: str) -> Union[int, float]:
number_as_float = float(value)
number_as_int = int(number_as_float)
return number_as_int if number_as_float == number_as_int else number_as_float
def strip_accents(s: Optional[str]) -> Optional[str]:
return s.translate(SYMBOL_TRANSLATION_MAP) if s is not None else s
def sanitize_comments_html(html: str) -> str:
text = html2text(html)
md = Markdown()
html = md.convert(text)
return html
def html2text(html: str) -> str:
# replace <u> tags with <span> as <u> becomes emphasis in html2text
if isinstance(html, bytes):
html = html.decode("utf-8")
html = re.sub(
r"<\s*(?P<solidus>/?)\s*[uU]\b(?P<rest>[^>]*)>",
r"<\g<solidus>span\g<rest>>",
html,
)
h2t = HTML2Text()
h2t.body_width = 0
h2t.single_line_break = True
h2t.emphasis_mark = "*"
return h2t.handle(html)
class LubimyCzytac(Metadata):
__name__ = "LubimyCzytac.pl"
__id__ = "lubimyczytac"
BASE_URL = "https://lubimyczytac.pl"
BOOK_SEARCH_RESULT_XPATH = (
"*//div[@class='listSearch']//div[@class='authorAllBooks__single']"
)
SINGLE_BOOK_RESULT_XPATH = ".//div[contains(@class,'authorAllBooks__singleText')]"
TITLE_PATH = "/div/a[contains(@class,'authorAllBooks__singleTextTitle')]"
TITLE_TEXT_PATH = f"{TITLE_PATH}//text()"
URL_PATH = f"{TITLE_PATH}/@href"
AUTHORS_PATH = "/div/a[contains(@href,'autor')]//text()"
SIBLINGS = "/following-sibling::dd"
CONTAINER = "//section[@class='container book']"
PUBLISHER = f"{CONTAINER}//dt[contains(text(),'Wydawnictwo:')]{SIBLINGS}/a/text()"
LANGUAGES = f"{CONTAINER}//dt[contains(text(),'Język:')]{SIBLINGS}/text()"
DESCRIPTION = f"{CONTAINER}//div[@class='collapse-content']"
SERIES = f"{CONTAINER}//span/a[contains(@href,'/cykl/')]/text()"
DETAILS = "//div[@id='book-details']"
PUBLISH_DATE = "//dt[contains(@title,'Data pierwszego wydania"
FIRST_PUBLISH_DATE = f"{DETAILS}{PUBLISH_DATE} oryginalnego')]{SIBLINGS}[1]/text()"
FIRST_PUBLISH_DATE_PL = f"{DETAILS}{PUBLISH_DATE} polskiego')]{SIBLINGS}[1]/text()"
TAGS = "//nav[@aria-label='breadcrumb']//a[contains(@href,'/ksiazki/k/')]/text()"
RATING = "//meta[@property='books:rating:value']/@content"
COVER = "//meta[@property='og:image']/@content"
ISBN = "//meta[@property='books:isbn']/@content"
META_TITLE = "//meta[@property='og:description']/@content"
SUMMARY = "//script[@type='application/ld+json']//text()"
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
if self.active:
try:
result = requests.get(self._prepare_query(title=query))
result.raise_for_status()
except Exception as e:
log.warning(e)
return None
root = fromstring(result.text)
lc_parser = LubimyCzytacParser(root=root, metadata=self)
matches = lc_parser.parse_search_results()
if matches:
with ThreadPool(processes=10) as pool:
final_matches = pool.starmap(
lc_parser.parse_single_book,
[(match, generic_cover, locale) for match in matches],
)
return final_matches
return matches
def _prepare_query(self, title: str) -> str:
query = ""
characters_to_remove = "\?()\/"
pattern = "[" + characters_to_remove + "]"
title = re.sub(pattern, "", title)
title = title.replace("_", " ")
if '"' in title or ",," in title:
title = title.split('"')[0].split(",,")[0]
if "/" in title:
title_tokens = [
token for token in title.lower().split(" ") if len(token) > 1
]
else:
title_tokens = list(self.get_title_tokens(title, strip_joiners=False))
if title_tokens:
tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = query + "%20".join(tokens)
if not query:
return ""
return f"{LubimyCzytac.BASE_URL}/szukaj/ksiazki?phrase={query}"
class LubimyCzytacParser:
PAGES_TEMPLATE = "<p id='strony'>Książka ma {0} stron(y).</p>"
PUBLISH_DATE_TEMPLATE = "<p id='pierwsze_wydanie'>Data pierwszego wydania: {0}</p>"
PUBLISH_DATE_PL_TEMPLATE = (
"<p id='pierwsze_wydanie'>Data pierwszego wydania w Polsce: {0}</p>"
)
def __init__(self, root: HtmlElement, metadata: Metadata) -> None:
self.root = root
self.metadata = metadata
def parse_search_results(self) -> List[MetaRecord]:
matches = []
results = self.root.xpath(LubimyCzytac.BOOK_SEARCH_RESULT_XPATH)
for result in results:
title = self._parse_xpath_node(
root=result,
xpath=f"{LubimyCzytac.SINGLE_BOOK_RESULT_XPATH}"
f"{LubimyCzytac.TITLE_TEXT_PATH}",
)
book_url = self._parse_xpath_node(
root=result,
xpath=f"{LubimyCzytac.SINGLE_BOOK_RESULT_XPATH}"
f"{LubimyCzytac.URL_PATH}",
)
authors = self._parse_xpath_node(
root=result,
xpath=f"{LubimyCzytac.SINGLE_BOOK_RESULT_XPATH}"
f"{LubimyCzytac.AUTHORS_PATH}",
take_first=False,
)
if not all([title, book_url, authors]):
continue
matches.append(
MetaRecord(
id=book_url.replace(f"/ksiazka/", "").split("/")[0],
title=title,
authors=[strip_accents(author) for author in authors],
url=LubimyCzytac.BASE_URL + book_url,
source=MetaSourceInfo(
id=self.metadata.__id__,
description=self.metadata.__name__,
link=LubimyCzytac.BASE_URL,
),
)
)
return matches
def parse_single_book(
self, match: MetaRecord, generic_cover: str, locale: str
) -> MetaRecord:
try:
response = requests.get(match.url)
response.raise_for_status()
except Exception as e:
log.warning(e)
return None
self.root = fromstring(response.text)
match.cover = self._parse_cover(generic_cover=generic_cover)
match.description = self._parse_description()
match.languages = self._parse_languages(locale=locale)
match.publisher = self._parse_publisher()
match.publishedDate = self._parse_from_summary(attribute_name="datePublished")
match.rating = self._parse_rating()
match.series, match.series_index = self._parse_series()
match.tags = self._parse_tags()
match.identifiers = {
"isbn": self._parse_isbn(),
"lubimyczytac": match.id,
}
return match
def _parse_xpath_node(
self,
xpath: str,
root: HtmlElement = None,
take_first: bool = True,
strip_element: bool = True,
) -> Optional[Union[str, List[str]]]:
root = root if root is not None else self.root
node = root.xpath(xpath)
if not node:
return None
return (
(node[0].strip() if strip_element else node[0])
if take_first
else [x.strip() for x in node]
)
def _parse_cover(self, generic_cover) -> Optional[str]:
return (
self._parse_xpath_node(xpath=LubimyCzytac.COVER, take_first=True)
or generic_cover
)
def _parse_publisher(self) -> Optional[str]:
return self._parse_xpath_node(xpath=LubimyCzytac.PUBLISHER, take_first=True)
def _parse_languages(self, locale: str) -> List[str]:
languages = list()
lang = self._parse_xpath_node(xpath=LubimyCzytac.LANGUAGES, take_first=True)
if lang:
if "polski" in lang:
languages.append("pol")
if "angielski" in lang:
languages.append("eng")
return [get_language_name(locale, language) for language in languages]
def _parse_series(self) -> Tuple[Optional[str], Optional[Union[float, int]]]:
series_index = 0
series = self._parse_xpath_node(xpath=LubimyCzytac.SERIES, take_first=True)
if series:
if "tom " in series:
series_name, series_info = series.split(" (tom ", 1)
series_info = series_info.replace(" ", "").replace(")", "")
# Check if book is not a bundle, i.e. chapter 1-3
if "-" in series_info:
series_info = series_info.split("-", 1)[0]
if series_info.replace(".", "").isdigit() is True:
series_index = get_int_or_float(series_info)
return series_name, series_index
return None, None
def _parse_tags(self) -> List[str]:
tags = self._parse_xpath_node(xpath=LubimyCzytac.TAGS, take_first=False)
return [
strip_accents(w.replace(", itd.", " itd."))
for w in tags
if isinstance(w, str)
]
def _parse_from_summary(self, attribute_name: str) -> Optional[str]:
value = None
summary_text = self._parse_xpath_node(xpath=LubimyCzytac.SUMMARY)
if summary_text:
data = json.loads(summary_text)
value = data.get(attribute_name)
return value.strip() if value is not None else value
def _parse_rating(self) -> Optional[str]:
rating = self._parse_xpath_node(xpath=LubimyCzytac.RATING)
return round(float(rating.replace(",", ".")) / 2) if rating else rating
def _parse_date(self, xpath="first_publish") -> Optional[datetime.datetime]:
options = {
"first_publish": LubimyCzytac.FIRST_PUBLISH_DATE,
"first_publish_pl": LubimyCzytac.FIRST_PUBLISH_DATE_PL,
}
date = self._parse_xpath_node(xpath=options.get(xpath))
return parser.parse(date) if date else None
def _parse_isbn(self) -> Optional[str]:
return self._parse_xpath_node(xpath=LubimyCzytac.ISBN)
def _parse_description(self) -> str:
description = ""
description_node = self._parse_xpath_node(
xpath=LubimyCzytac.DESCRIPTION, strip_element=False
)
if description_node is not None:
for source in self.root.xpath('//p[@class="source"]'):
source.getparent().remove(source)
description = tostring(description_node, method="html")
description = sanitize_comments_html(description)
else:
description_node = self._parse_xpath_node(xpath=LubimyCzytac.META_TITLE)
if description_node is not None:
description = description_node
description = sanitize_comments_html(description)
description = self._add_extra_info_to_description(description=description)
return description
def _add_extra_info_to_description(self, description: str) -> str:
pages = self._parse_from_summary(attribute_name="numberOfPages")
if pages:
description += LubimyCzytacParser.PAGES_TEMPLATE.format(pages)
first_publish_date = self._parse_date()
if first_publish_date:
description += LubimyCzytacParser.PUBLISH_DATE_TEMPLATE.format(
first_publish_date.strftime("%d.%m.%Y")
)
first_publish_date_pl = self._parse_date(xpath="first_publish_pl")
if first_publish_date_pl:
description += LubimyCzytacParser.PUBLISH_DATE_PL_TEMPLATE.format(
first_publish_date_pl.strftime("%d.%m.%Y")
)
return description

84
cps/metadata_provider/scholar.py
View File

@ -15,47 +15,69 @@
# #
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import itertools
from typing import Dict, List, Optional
from urllib.parse import quote, unquote
from scholarly import scholarly try:
from fake_useragent.errors import FakeUserAgentError
except (ImportError):
FakeUserAgentError = BaseException
try:
from scholarly import scholarly
except FakeUserAgentError:
raise ImportError("No module named 'scholarly'")
from cps.services.Metadata import Metadata from cps import logger
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
class scholar(Metadata): class scholar(Metadata):
__name__ = "Google Scholar" __name__ = "Google Scholar"
__id__ = "googlescholar" __id__ = "googlescholar"
META_URL = "https://scholar.google.com/"
def search(self, query, generic_cover=""): def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
val = list() val = list()
if self.active: if self.active:
scholar_gen = scholarly.search_pubs(' '.join(query.split('+'))) title_tokens = list(self.get_title_tokens(query, strip_joiners=False))
i = 0 if title_tokens:
for publication in scholar_gen: tokens = [quote(t.encode("utf-8")) for t in title_tokens]
v = dict() query = " ".join(tokens)
v['id'] = "1234" # publication['bib'].get('title') try:
v['title'] = publication['bib'].get('title') scholarly.set_timeout(20)
v['authors'] = publication['bib'].get('author', []) scholarly.set_retries(2)
v['description'] = publication['bib'].get('abstract', "") scholar_gen = itertools.islice(scholarly.search_pubs(query), 10)
v['publisher'] = publication['bib'].get('venue', "") except Exception as e:
if publication['bib'].get('pub_year'): log.warning(e)
v['publishedDate'] = publication['bib'].get('pub_year')+"-01-01" return list()
else: for result in scholar_gen:
v['publishedDate'] = "" match = self._parse_search_result(
v['tags'] = "" result=result, generic_cover="", locale=locale
v['ratings'] = 0 )
v['series'] = "" val.append(match)
v['cover'] = generic_cover
v['url'] = publication.get('pub_url') or publication.get('eprint_url') or "",
v['source'] = {
"id": self.__id__,
"description": "Google Scholar",
"link": "https://scholar.google.com/"
}
val.append(v)
i += 1
if (i >= 10):
break
return val return val
def _parse_search_result(
self, result: Dict, generic_cover: str, locale: str
) -> MetaRecord:
match = MetaRecord(
id=result.get("pub_url", result.get("eprint_url", "")),
title=result["bib"].get("title"),
authors=result["bib"].get("author", []),
url=result.get("pub_url", result.get("eprint_url", "")),
source=MetaSourceInfo(
id=self.__id__, description=self.__name__, link=scholar.META_URL
),
)
match.cover = result.get("image", {}).get("original_url", generic_cover)
match.description = unquote(result["bib"].get("abstract", ""))
match.publisher = result["bib"].get("venue", "")
match.publishedDate = result["bib"].get("pub_year") + "-01-01"
match.identifiers = {"scholar": match.id}
return match

14
cps/oauth.py
View File

@ -19,18 +19,12 @@
from flask import session from flask import session
try: try:
from flask_dance.consumer.backend.sqla import SQLAlchemyBackend, first, _get_real_user from flask_dance.consumer.storage.sqla import SQLAlchemyStorage as SQLAlchemyBackend
from flask_dance.consumer.storage.sqla import first, _get_real_user
from sqlalchemy.orm.exc import NoResultFound from sqlalchemy.orm.exc import NoResultFound
backend_resultcode = False # prevent storing values with this resultcode backend_resultcode = True # prevent storing values with this resultcode
except ImportError: except ImportError:
# fails on flask-dance >1.3, due to renaming pass
try:
from flask_dance.consumer.storage.sqla import SQLAlchemyStorage as SQLAlchemyBackend
from flask_dance.consumer.storage.sqla import first, _get_real_user
from sqlalchemy.orm.exc import NoResultFound
backend_resultcode = True # prevent storing values with this resultcode
except ImportError:
pass
class OAuthBackend(SQLAlchemyBackend): class OAuthBackend(SQLAlchemyBackend):

46
cps/oauth_bb.py
View File

@ -74,7 +74,7 @@ def register_user_with_oauth(user=None):
if len(all_oauth.keys()) == 0: if len(all_oauth.keys()) == 0:
return return
if user is None: if user is None:
flash(_(u"Register with %(provider)s", provider=", ".join(list(all_oauth.values()))), category="success") flash(_("Register with %(provider)s", provider=", ".join(list(all_oauth.values()))), category="success")
else: else:
for oauth_key in all_oauth.keys(): for oauth_key in all_oauth.keys():
# Find this OAuth token in the database, or create it # Find this OAuth token in the database, or create it
@ -134,8 +134,8 @@ def bind_oauth_or_register(provider_id, provider_user_id, redirect_url, provider
# already bind with user, just login # already bind with user, just login
if oauth_entry.user: if oauth_entry.user:
login_user(oauth_entry.user) login_user(oauth_entry.user)
log.debug(u"You are now logged in as: '%s'", oauth_entry.user.name) log.debug("You are now logged in as: '%s'", oauth_entry.user.name)
flash(_(u"you are now logged in as: '%(nickname)s'", nickname= oauth_entry.user.name), flash(_("Success! You are now logged in as: %(nickname)s", nickname= oauth_entry.user.name),
category="success") category="success")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
else: else:
@ -145,21 +145,21 @@ def bind_oauth_or_register(provider_id, provider_user_id, redirect_url, provider
try: try:
ub.session.add(oauth_entry) ub.session.add(oauth_entry)
ub.session.commit() ub.session.commit()
flash(_(u"Link to %(oauth)s Succeeded", oauth=provider_name), category="success") flash(_("Link to %(oauth)s Succeeded", oauth=provider_name), category="success")
log.info("Link to {} Succeeded".format(provider_name)) log.info("Link to {} Succeeded".format(provider_name))
return redirect(url_for('web.profile')) return redirect(url_for('web.profile'))
except Exception as ex: except Exception as ex:
log.debug_or_exception(ex) log.error_or_exception(ex)
ub.session.rollback() ub.session.rollback()
else: else:
flash(_(u"Login failed, No User Linked With OAuth Account"), category="error") flash(_("Login failed, No User Linked With OAuth Account"), category="error")
log.info('Login failed, No User Linked With OAuth Account') log.info('Login failed, No User Linked With OAuth Account')
return redirect(url_for('web.login')) return redirect(url_for('web.login'))
# return redirect(url_for('web.login')) # return redirect(url_for('web.login'))
# if config.config_public_reg: # if config.config_public_reg:
# return redirect(url_for('web.register')) # return redirect(url_for('web.register'))
# else: # else:
# flash(_(u"Public registration is not enabled"), category="error") # flash(_("Public registration is not enabled"), category="error")
# return redirect(url_for(redirect_url)) # return redirect(url_for(redirect_url))
except (NoResultFound, AttributeError): except (NoResultFound, AttributeError):
return redirect(url_for(redirect_url)) return redirect(url_for(redirect_url))
@ -194,15 +194,15 @@ def unlink_oauth(provider):
ub.session.delete(oauth_entry) ub.session.delete(oauth_entry)
ub.session.commit() ub.session.commit()
logout_oauth_user() logout_oauth_user()
flash(_(u"Unlink to %(oauth)s Succeeded", oauth=oauth_check[provider]), category="success") flash(_("Unlink to %(oauth)s Succeeded", oauth=oauth_check[provider]), category="success")
log.info("Unlink to {} Succeeded".format(oauth_check[provider])) log.info("Unlink to {} Succeeded".format(oauth_check[provider]))
except Exception as ex: except Exception as ex:
log.debug_or_exception(ex) log.error_or_exception(ex)
ub.session.rollback() ub.session.rollback()
flash(_(u"Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error") flash(_("Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error")
except NoResultFound: except NoResultFound:
log.warning("oauth %s for user %d not found", provider, current_user.id) log.warning("oauth %s for user %d not found", provider, current_user.id)
flash(_(u"Not Linked to %(oauth)s", oauth=provider), category="error") flash(_("Not Linked to %(oauth)s", oauth=provider), category="error")
return redirect(url_for('web.profile')) return redirect(url_for('web.profile'))
def generate_oauth_blueprints(): def generate_oauth_blueprints():
@ -258,13 +258,13 @@ if ub.oauth_support:
@oauth_authorized.connect_via(oauthblueprints[0]['blueprint']) @oauth_authorized.connect_via(oauthblueprints[0]['blueprint'])
def github_logged_in(blueprint, token): def github_logged_in(blueprint, token):
if not token: if not token:
flash(_(u"Failed to log in with GitHub."), category="error") flash(_("Failed to log in with GitHub."), category="error")
log.error("Failed to log in with GitHub") log.error("Failed to log in with GitHub")
return False return False
resp = blueprint.session.get("/user") resp = blueprint.session.get("/user")
if not resp.ok: if not resp.ok:
flash(_(u"Failed to fetch user info from GitHub."), category="error") flash(_("Failed to fetch user info from GitHub."), category="error")
log.error("Failed to fetch user info from GitHub") log.error("Failed to fetch user info from GitHub")
return False return False
@ -276,13 +276,13 @@ if ub.oauth_support:
@oauth_authorized.connect_via(oauthblueprints[1]['blueprint']) @oauth_authorized.connect_via(oauthblueprints[1]['blueprint'])
def google_logged_in(blueprint, token): def google_logged_in(blueprint, token):
if not token: if not token:
flash(_(u"Failed to log in with Google."), category="error") flash(_("Failed to log in with Google."), category="error")
log.error("Failed to log in with Google") log.error("Failed to log in with Google")
return False return False
resp = blueprint.session.get("/oauth2/v2/userinfo") resp = blueprint.session.get("/oauth2/v2/userinfo")
if not resp.ok: if not resp.ok:
flash(_(u"Failed to fetch user info from Google."), category="error") flash(_("Failed to fetch user info from Google."), category="error")
log.error("Failed to fetch user info from Google") log.error("Failed to fetch user info from Google")
return False return False
@ -295,8 +295,8 @@ if ub.oauth_support:
@oauth_error.connect_via(oauthblueprints[0]['blueprint']) @oauth_error.connect_via(oauthblueprints[0]['blueprint'])
def github_error(blueprint, error, error_description=None, error_uri=None): def github_error(blueprint, error, error_description=None, error_uri=None):
msg = ( msg = (
u"OAuth error from {name}! " "OAuth error from {name}! "
u"error={error} description={description} uri={uri}" "error={error} description={description} uri={uri}"
).format( ).format(
name=blueprint.name, name=blueprint.name,
error=error, error=error,
@ -308,8 +308,8 @@ if ub.oauth_support:
@oauth_error.connect_via(oauthblueprints[1]['blueprint']) @oauth_error.connect_via(oauthblueprints[1]['blueprint'])
def google_error(blueprint, error, error_description=None, error_uri=None): def google_error(blueprint, error, error_description=None, error_uri=None):
msg = ( msg = (
u"OAuth error from {name}! " "OAuth error from {name}! "
u"error={error} description={description} uri={uri}" "error={error} description={description} uri={uri}"
).format( ).format(
name=blueprint.name, name=blueprint.name,
error=error, error=error,
@ -329,10 +329,10 @@ def github_login():
if account_info.ok: if account_info.ok:
account_info_json = account_info.json() account_info_json = account_info.json()
return bind_oauth_or_register(oauthblueprints[0]['id'], account_info_json['id'], 'github.login', 'github') return bind_oauth_or_register(oauthblueprints[0]['id'], account_info_json['id'], 'github.login', 'github')
flash(_(u"GitHub Oauth error, please retry later."), category="error") flash(_("GitHub Oauth error, please retry later."), category="error")
log.error("GitHub Oauth error, please retry later") log.error("GitHub Oauth error, please retry later")
except (InvalidGrantError, TokenExpiredError) as e: except (InvalidGrantError, TokenExpiredError) as e:
flash(_(u"GitHub Oauth error: {}").format(e), category="error") flash(_("GitHub Oauth error: {}").format(e), category="error")
log.error(e) log.error(e)
return redirect(url_for('web.login')) return redirect(url_for('web.login'))
@ -353,10 +353,10 @@ def google_login():
if resp.ok: if resp.ok:
account_info_json = resp.json() account_info_json = resp.json()
return bind_oauth_or_register(oauthblueprints[1]['id'], account_info_json['id'], 'google.login', 'google') return bind_oauth_or_register(oauthblueprints[1]['id'], account_info_json['id'], 'google.login', 'google')
flash(_(u"Google Oauth error, please retry later."), category="error") flash(_("Google Oauth error, please retry later."), category="error")
log.error("Google Oauth error, please retry later") log.error("Google Oauth error, please retry later")
except (InvalidGrantError, TokenExpiredError) as e: except (InvalidGrantError, TokenExpiredError) as e:
flash(_(u"Google Oauth error: {}").format(e), category="error") flash(_("Google Oauth error: {}").format(e), category="error")
log.error(e) log.error(e)
return redirect(url_for('web.login')) return redirect(url_for('web.login'))

354
cps/opds.py
View File

@ -20,56 +20,28 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import datetime import datetime
from functools import wraps from urllib.parse import unquote_plus
from flask import Blueprint, request, render_template, Response, g, make_response, abort from flask import Blueprint, request, render_template, make_response, abort
from flask_login import current_user from flask_login import current_user
from flask_babel import get_locale
from flask_babel import gettext as _
from sqlalchemy.sql.expression import func, text, or_, and_, true from sqlalchemy.sql.expression import func, text, or_, and_, true
from werkzeug.security import check_password_hash from sqlalchemy.exc import InvalidRequestError, OperationalError
from . import constants, logger, config, db, calibre_db, ub, services, get_locale, isoLanguages from . import logger, config, db, calibre_db, ub, isoLanguages
from .usermanagement import requires_basic_auth_if_no_ano
from .helper import get_download_link, get_book_cover from .helper import get_download_link, get_book_cover
from .pagination import Pagination from .pagination import Pagination
from .web import render_read_books from .web import render_read_books
from .usermanagement import load_user_from_request
from flask_babel import gettext as _
from babel import Locale as LC
from babel.core import UnknownLocaleError
opds = Blueprint('opds', __name__) opds = Blueprint('opds', __name__)
log = logger.create() log = logger.create()
def requires_basic_auth_if_no_ano(f):
@wraps(f)
def decorated(*args, **kwargs):
auth = request.authorization
if config.config_anonbrowse != 1:
if not auth or auth.type != 'basic' or not check_auth(auth.username, auth.password):
return authenticate()
return f(*args, **kwargs)
if config.config_login_type == constants.LOGIN_LDAP and services.ldap and config.config_anonbrowse != 1:
return services.ldap.basic_auth_required(f)
return decorated
class FeedObject:
def __init__(self, rating_id, rating_name):
self.rating_id = rating_id
self.rating_name = rating_name
@property
def id(self):
return self.rating_id
@property
def name(self):
return self.rating_name
@opds.route("/opds/") @opds.route("/opds/")
@opds.route("/opds") @opds.route("/opds")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
@ -84,10 +56,12 @@ def feed_osd():
@opds.route("/opds/search", defaults={'query': ""}) @opds.route("/opds/search", defaults={'query': ""})
@opds.route("/opds/search/<query>") @opds.route("/opds/search/<path:query>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_cc_search(query): def feed_cc_search(query):
return feed_search(query.strip()) # Handle strange query from Libera Reader with + instead of spaces
plus_query = unquote_plus(request.environ['RAW_URI'].split('/opds/search/')[1]).strip()
return feed_search(plus_query)
@opds.route("/opds/search", methods=["GET"]) @opds.route("/opds/search", methods=["GET"])
@ -99,26 +73,7 @@ def feed_normal_search():
@opds.route("/opds/books") @opds.route("/opds/books")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_booksindex(): def feed_booksindex():
shift = 0 return render_element_index(db.Books.sort, None, 'opds.feed_letter_books')
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Books.sort, 1, 1)).label('id'))\
.filter(calibre_db.common_filters()).group_by(func.upper(func.substr(db.Books.sort, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_books',
pagination=pagination)
@opds.route("/opds/books/letter/<book_id>") @opds.route("/opds/books/letter/<book_id>")
@ -129,7 +84,8 @@ def feed_letter_books(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0, entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, db.Books,
letter, letter,
[db.Books.sort]) [db.Books.sort],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -139,15 +95,16 @@ def feed_letter_books(book_id):
def feed_new(): def feed_new():
off = request.args.get("offset") or 0 off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0, entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, True, [db.Books.timestamp.desc()]) db.Books, True, [db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/discover") @opds.route("/opds/discover")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_discover(): def feed_discover():
entries = calibre_db.session.query(db.Books).filter(calibre_db.common_filters()).order_by(func.random())\ query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
.limit(config.config_books_per_page) entries = query.filter(calibre_db.common_filters()).order_by(func.random()).limit(config.config_books_per_page)
pagination = Pagination(1, config.config_books_per_page, int(config.config_books_per_page)) pagination = Pagination(1, config.config_books_per_page, int(config.config_books_per_page))
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -158,7 +115,8 @@ def feed_best_rated():
off = request.args.get("offset") or 0 off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0, entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, db.Books.ratings.any(db.Ratings.rating > 9), db.Books, db.Books.ratings.any(db.Ratings.rating > 9),
[db.Books.timestamp.desc()]) [db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -171,43 +129,23 @@ def feed_hot():
hot_books = all_books.offset(off).limit(config.config_books_per_page) hot_books = all_books.offset(off).limit(config.config_books_per_page)
entries = list() entries = list()
for book in hot_books: for book in hot_books:
downloadBook = calibre_db.get_book(book.Downloads.book_id) query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
if downloadBook: download_book = query.filter(calibre_db.common_filters()).filter(
entries.append( book.Downloads.book_id == db.Books.id).first()
calibre_db.get_filtered_book(book.Downloads.book_id) if download_book:
) entries.append(download_book)
else: else:
ub.delete_download(book.Downloads.book_id) ub.delete_download(book.Downloads.book_id)
numBooks = entries.__len__() num_books = entries.__len__()
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1),
config.config_books_per_page, numBooks) config.config_books_per_page, num_books)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/author") @opds.route("/opds/author")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_authorindex(): def feed_authorindex():
shift = 0 return render_element_index(db.Authors.sort, db.books_authors_link, 'opds.feed_letter_author')
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Authors.sort, 1, 1)).label('id'))\
.join(db.books_authors_link).join(db.Books).filter(calibre_db.common_filters())\
.group_by(func.upper(func.substr(db.Authors.sort, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_author',
pagination=pagination)
@opds.route("/opds/author/letter/<book_id>") @opds.route("/opds/author/letter/<book_id>")
@ -228,12 +166,7 @@ def feed_letter_author(book_id):
@opds.route("/opds/author/<int:book_id>") @opds.route("/opds/author/<int:book_id>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_author(book_id): def feed_author(book_id):
off = request.args.get("offset") or 0 return render_xml_dataset(db.Authors, book_id)
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.authors.any(db.Authors.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/publisher") @opds.route("/opds/publisher")
@ -254,37 +187,14 @@ def feed_publisherindex():
@opds.route("/opds/publisher/<int:book_id>") @opds.route("/opds/publisher/<int:book_id>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_publisher(book_id): def feed_publisher(book_id):
off = request.args.get("offset") or 0 return render_xml_dataset(db.Publishers, book_id)
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.publishers.any(db.Publishers.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/category") @opds.route("/opds/category")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_categoryindex(): def feed_categoryindex():
shift = 0 return render_element_index(db.Tags.name, db.books_tags_link, 'opds.feed_letter_category')
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Tags.name, 1, 1)).label('id'))\
.join(db.books_tags_link).join(db.Books).filter(calibre_db.common_filters())\
.group_by(func.upper(func.substr(db.Tags.name, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_category',
pagination=pagination)
@opds.route("/opds/category/letter/<book_id>") @opds.route("/opds/category/letter/<book_id>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
@ -306,36 +216,14 @@ def feed_letter_category(book_id):
@opds.route("/opds/category/<int:book_id>") @opds.route("/opds/category/<int:book_id>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_category(book_id): def feed_category(book_id):
off = request.args.get("offset") or 0 return render_xml_dataset(db.Tags, book_id)
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.tags.any(db.Tags.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/series") @opds.route("/opds/series")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_seriesindex(): def feed_seriesindex():
shift = 0 return render_element_index(db.Series.sort, db.books_series_link, 'opds.feed_letter_series')
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Series.sort, 1, 1)).label('id'))\
.join(db.books_series_link).join(db.Books).filter(calibre_db.common_filters())\
.group_by(func.upper(func.substr(db.Series.sort, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_series',
pagination=pagination)
@opds.route("/opds/series/letter/<book_id>") @opds.route("/opds/series/letter/<book_id>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
@ -361,7 +249,8 @@ def feed_series(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0, entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, db.Books,
db.Books.series.any(db.Series.id == book_id), db.Books.series.any(db.Series.id == book_id),
[db.Books.series_index]) [db.Books.series_index],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -370,7 +259,7 @@ def feed_series(book_id):
def feed_ratingindex(): def feed_ratingindex():
off = request.args.get("offset") or 0 off = request.args.get("offset") or 0
entries = calibre_db.session.query(db.Ratings, func.count('books_ratings_link.book').label('count'), entries = calibre_db.session.query(db.Ratings, func.count('books_ratings_link.book').label('count'),
(db.Ratings.rating / 2).label('name')) \ (db.Ratings.rating / 2).label('name')) \
.join(db.books_ratings_link)\ .join(db.books_ratings_link)\
.join(db.Books)\ .join(db.Books)\
.filter(calibre_db.common_filters()) \ .filter(calibre_db.common_filters()) \
@ -388,12 +277,7 @@ def feed_ratingindex():
@opds.route("/opds/ratings/<book_id>") @opds.route("/opds/ratings/<book_id>")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_ratings(book_id): def feed_ratings(book_id):
off = request.args.get("offset") or 0 return render_xml_dataset(db.Ratings, book_id)
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.ratings.any(db.Ratings.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/formats") @opds.route("/opds/formats")
@ -420,7 +304,8 @@ def feed_format(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0, entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, db.Books,
db.Books.data.any(db.Data.format == book_id.upper()), db.Books.data.any(db.Data.format == book_id.upper()),
[db.Books.timestamp.desc()]) [db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -429,20 +314,12 @@ def feed_format(book_id):
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_languagesindex(): def feed_languagesindex():
off = request.args.get("offset") or 0 off = request.args.get("offset") or 0
if current_user.filter_language() == u"all": if current_user.filter_language() == "all":
languages = calibre_db.speaking_language() languages = calibre_db.speaking_language()
else: else:
#try:
# cur_l = LC.parse(current_user.filter_language())
#except UnknownLocaleError:
# cur_l = None
languages = calibre_db.session.query(db.Languages).filter( languages = calibre_db.session.query(db.Languages).filter(
db.Languages.lang_code == current_user.filter_language()).all() db.Languages.lang_code == current_user.filter_language()).all()
languages[0].name = isoLanguages.get_language_name(get_locale(), languages[0].lang_code) languages[0].name = isoLanguages.get_language_name(get_locale(), languages[0].lang_code)
#if cur_l:
# languages[0].name = cur_l.get_language_name(get_locale())
#else:
# languages[0].name = _(isoLanguages.get(part3=languages[0].lang_code).name)
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page, pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(languages)) len(languages))
return render_xml_template('feed.xml', listelements=languages, folder='opds.feed_languages', pagination=pagination) return render_xml_template('feed.xml', listelements=languages, folder='opds.feed_languages', pagination=pagination)
@ -455,7 +332,8 @@ def feed_languages(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0, entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, db.Books,
db.Books.languages.any(db.Languages.id == book_id), db.Books.languages.any(db.Languages.id == book_id),
[db.Books.timestamp.desc()]) [db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination) return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -463,7 +341,8 @@ def feed_languages(book_id):
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def feed_shelfindex(): def feed_shelfindex():
off = request.args.get("offset") or 0 off = request.args.get("offset") or 0
shelf = g.shelves_access shelf = ub.session.query(ub.Shelf).filter(
or_(ub.Shelf.is_public == 1, ub.Shelf.user_id == current_user.id)).order_by(ub.Shelf.name).all()
number = len(shelf) number = len(shelf)
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page, pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
number) number)
@ -485,24 +364,32 @@ def feed_shelf(book_id):
result = list() result = list()
# user is allowed to access shelf # user is allowed to access shelf
if shelf: if shelf:
books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == book_id).order_by( result, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
ub.BookShelf.order.asc()).all() config.config_books_per_page,
for book in books_in_shelf: db.Books,
cur_book = calibre_db.get_book(book.book_id) ub.BookShelf.shelf == shelf.id,
result.append(cur_book) [ub.BookShelf.order.asc()],
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page, True, config.config_read_column,
len(result)) ub.BookShelf, ub.BookShelf.book_id == db.Books.id)
# delete shelf entries where book is not existent anymore, can happen if book is deleted outside calibre-web
wrong_entries = calibre_db.session.query(ub.BookShelf) \
.join(db.Books, ub.BookShelf.book_id == db.Books.id, isouter=True) \
.filter(db.Books.id == None).all()
for entry in wrong_entries:
log.info('Not existing book {} in {} deleted'.format(entry.book_id, shelf))
try:
ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == entry.book_id).delete()
ub.session.commit()
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error_or_exception("Settings Database error: {}".format(e))
return render_xml_template('feed.xml', entries=result, pagination=pagination) return render_xml_template('feed.xml', entries=result, pagination=pagination)
@opds.route("/opds/download/<book_id>/<book_format>/") @opds.route("/opds/download/<book_id>/<book_format>/")
@requires_basic_auth_if_no_ano @requires_basic_auth_if_no_ano
def opds_download_link(book_id, book_format): def opds_download_link(book_id, book_format):
# I gave up with this: With enabled ldap login, the user doesn't get logged in, therefore it's always guest if not current_user.role_download():
# workaround, loading the user from the request and checking it's download rights here
# in case of anonymous browsing user is None
user = load_user_from_request(request) or current_user
if not user.role_download():
return abort(403) return abort(403)
if "Kobo" in request.headers.get('User-Agent'): if "Kobo" in request.headers.get('User-Agent'):
client = "kobo" client = "kobo"
@ -525,47 +412,6 @@ def get_metadata_calibre_companion(uuid, library):
return "" return ""
def feed_search(term):
if term:
entries, __, ___ = calibre_db.get_search_results(term)
entriescount = len(entries) if len(entries) > 0 else 1
pagination = Pagination(1, entriescount, entriescount)
return render_xml_template('feed.xml', searchterm=term, entries=entries, pagination=pagination)
else:
return render_xml_template('feed.xml', searchterm="")
def check_auth(username, password):
try:
username = username.encode('windows-1252')
except UnicodeEncodeError:
username = username.encode('utf-8')
user = ub.session.query(ub.User).filter(func.lower(ub.User.name) ==
username.decode('utf-8').lower()).first()
if bool(user and check_password_hash(str(user.password), password)):
return True
else:
ip_Address = request.headers.get('X-Forwarded-For', request.remote_addr)
log.warning('OPDS Login failed for user "%s" IP-address: %s', username.decode('utf-8'), ip_Address)
return False
def authenticate():
return Response(
'Could not verify your access level for that URL.\n'
'You have to login with proper credentials', 401,
{'WWW-Authenticate': 'Basic realm="Login Required"'})
def render_xml_template(*args, **kwargs):
# ToDo: return time in current timezone similar to %z
currtime = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%S+00:00")
xml = render_template(current_time=currtime, instance=config.config_calibre_web_title, *args, **kwargs)
response = make_response(xml)
response.headers["Content-Type"] = "application/atom+xml; charset=utf-8"
return response
@opds.route("/opds/thumb_240_240/<book_id>") @opds.route("/opds/thumb_240_240/<book_id>")
@opds.route("/opds/cover_240_240/<book_id>") @opds.route("/opds/cover_240_240/<book_id>")
@opds.route("/opds/cover_90_90/<book_id>") @opds.route("/opds/cover_90_90/<book_id>")
@ -589,3 +435,71 @@ def feed_unread_books():
off = request.args.get("offset") or 0 off = request.args.get("offset") or 0
result, pagination = render_read_books(int(off) / (int(config.config_books_per_page)) + 1, False, True) result, pagination = render_read_books(int(off) / (int(config.config_books_per_page)) + 1, False, True)
return render_xml_template('feed.xml', entries=result, pagination=pagination) return render_xml_template('feed.xml', entries=result, pagination=pagination)
class FeedObject:
def __init__(self, rating_id, rating_name):
self.rating_id = rating_id
self.rating_name = rating_name
@property
def id(self):
return self.rating_id
@property
def name(self):
return self.rating_name
def feed_search(term):
if term:
entries, __, ___ = calibre_db.get_search_results(term, config=config)
entries_count = len(entries) if len(entries) > 0 else 1
pagination = Pagination(1, entries_count, entries_count)
return render_xml_template('feed.xml', searchterm=term, entries=entries, pagination=pagination)
else:
return render_xml_template('feed.xml', searchterm="")
def render_xml_template(*args, **kwargs):
# ToDo: return time in current timezone similar to %z
currtime = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%S+00:00")
xml = render_template(current_time=currtime, instance=config.config_calibre_web_title, *args, **kwargs)
response = make_response(xml)
response.headers["Content-Type"] = "application/atom+xml; charset=utf-8"
return response
def render_xml_dataset(data_table, book_id):
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
getattr(db.Books, data_table.__tablename__).any(data_table.id == book_id),
[db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
def render_element_index(database_column, linked_table, folder):
shift = 0
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(database_column, 1, 1)).label('id'), None, None)
# query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
if linked_table is not None:
entries = entries.join(linked_table).join(db.Books)
entries = entries.filter(calibre_db.common_filters()).group_by(func.upper(func.substr(database_column, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name': _("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder=folder,
pagination=pagination)

8
cps/pagination.py
View File

@ -57,10 +57,10 @@ class Pagination(object):
def has_next(self): def has_next(self):
return self.page < self.pages return self.page < self.pages
# right_edge: last right_edges count of all pages are shown as number, means, if 10 pages are paginated -> 9,10 shwn # right_edge: last right_edges count of all pages are shown as number, means, if 10 pages are paginated -> 9,10 shown
# left_edge: first left_edges count of all pages are shown as number -> 1,2 shwn # left_edge: first left_edges count of all pages are shown as number -> 1,2 shown
# left_current: left_current count below current page are shown as number, means if current page 5 -> 3,4 shwn # left_current: left_current count below current page are shown as number, means if current page 5 -> 3,4 shown
# left_current: right_current count above current page are shown as number, means if current page 5 -> 6,7 shwn # left_current: right_current count above current page are shown as number, means if current page 5 -> 6,7 shown
def iter_pages(self, left_edge=2, left_current=2, def iter_pages(self, left_edge=2, left_current=2,
right_current=4, right_edge=2): right_current=4, right_edge=2):
last = 0 last = 0

7
cps/redirect.py
View File

@ -25,12 +25,9 @@
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE AND DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # IN ANY WAY OUT OF THE USE OF THIS SOFTWARE AND DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# http://flask.pocoo.org/snippets/62/ # https://web.archive.org/web/20120517003641/http://flask.pocoo.org/snippets/62/
try: from urllib.parse import urlparse, urljoin
from urllib.parse import urlparse, urljoin
except ImportError:
from urlparse import urlparse, urljoin
from flask import request, url_for, redirect from flask import request, url_for, redirect

29
cps/remotelogin.py
View File

@ -22,6 +22,7 @@
import json import json
from datetime import datetime from datetime import datetime
from functools import wraps
from flask import Blueprint, request, make_response, abort, url_for, flash, redirect from flask import Blueprint, request, make_response, abort, url_for, flash, redirect
from flask_login import login_required, current_user, login_user from flask_login import login_required, current_user, login_user
@ -31,10 +32,6 @@ from sqlalchemy.sql.expression import true
from . import config, logger, ub from . import config, logger, ub
from .render_template import render_title_template from .render_template import render_title_template
try:
from functools import wraps
except ImportError:
pass # We're not using Python 3
remotelogin = Blueprint('remotelogin', __name__) remotelogin = Blueprint('remotelogin', __name__)
log = logger.create() log = logger.create()
@ -61,8 +58,8 @@ def remote_login():
ub.session.add(auth_token) ub.session.add(auth_token)
ub.session_commit() ub.session_commit()
verify_url = url_for('remotelogin.verify_token', token=auth_token.auth_token, _external=true) verify_url = url_for('remotelogin.verify_token', token=auth_token.auth_token, _external=true)
log.debug(u"Remot Login request with token: %s", auth_token.auth_token) log.debug("Remot Login request with token: %s", auth_token.auth_token)
return render_title_template('remote_login.html', title=_(u"Login"), token=auth_token.auth_token, return render_title_template('remote_login.html', title=_("Login"), token=auth_token.auth_token,
verify_url=verify_url, page="remotelogin") verify_url=verify_url, page="remotelogin")
@ -74,8 +71,8 @@ def verify_token(token):
# Token not found # Token not found
if auth_token is None: if auth_token is None:
flash(_(u"Token not found"), category="error") flash(_("Token not found"), category="error")
log.error(u"Remote Login token not found") log.error("Remote Login token not found")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
# Token expired # Token expired
@ -83,8 +80,8 @@ def verify_token(token):
ub.session.delete(auth_token) ub.session.delete(auth_token)
ub.session_commit() ub.session_commit()
flash(_(u"Token has expired"), category="error") flash(_("Token has expired"), category="error")
log.error(u"Remote Login token expired") log.error("Remote Login token expired")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
# Update token with user information # Update token with user information
@ -92,8 +89,8 @@ def verify_token(token):
auth_token.verified = True auth_token.verified = True
ub.session_commit() ub.session_commit()
flash(_(u"Success! Please return to your device"), category="success") flash(_("Success! Please return to your device"), category="success")
log.debug(u"Remote Login token for userid %s verified", auth_token.user_id) log.debug("Remote Login token for userid %s verified", auth_token.user_id)
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
@ -108,7 +105,7 @@ def token_verified():
# Token not found # Token not found
if auth_token is None: if auth_token is None:
data['status'] = 'error' data['status'] = 'error'
data['message'] = _(u"Token not found") data['message'] = _("Token not found")
# Token expired # Token expired
elif datetime.now() > auth_token.expiration: elif datetime.now() > auth_token.expiration:
@ -116,7 +113,7 @@ def token_verified():
ub.session_commit() ub.session_commit()
data['status'] = 'error' data['status'] = 'error'
data['message'] = _(u"Token has expired") data['message'] = _("Token has expired")
elif not auth_token.verified: elif not auth_token.verified:
data['status'] = 'not_verified' data['status'] = 'not_verified'
@ -129,8 +126,8 @@ def token_verified():
ub.session_commit("User {} logged in via remotelogin, token deleted".format(user.name)) ub.session_commit("User {} logged in via remotelogin, token deleted".format(user.name))
data['status'] = 'success' data['status'] = 'success'
log.debug(u"Remote Login for userid %s succeded", user.id) log.debug("Remote Login for userid %s succeeded", user.id)
flash(_(u"you are now logged in as: '%(nickname)s'", nickname=user.name), category="success") flash(_("Success! You are now logged in as: %(nickname)s", nickname=user.name), category="success")
response = make_response(json.dumps(data, ensure_ascii=False)) response = make_response(json.dumps(data, ensure_ascii=False))
response.headers["Content-Type"] = "application/json; charset=utf-8" response.headers["Content-Type"] = "application/json; charset=utf-8"

77
cps/render_template.py
View File

@ -16,20 +16,23 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
from flask import render_template from flask import render_template, g, abort, request
from flask_babel import gettext as _ from flask_babel import gettext as _
from flask import g
from werkzeug.local import LocalProxy from werkzeug.local import LocalProxy
from flask_login import current_user from flask_login import current_user
from sqlalchemy.sql.expression import or_
from . import config, constants, ub, logger, db, calibre_db from . import config, constants, logger, ub
from .ub import User from .ub import User
log = logger.create() log = logger.create()
def get_sidebar_config(kwargs=None): def get_sidebar_config(kwargs=None):
kwargs = kwargs or [] kwargs = kwargs or []
simple = bool([e for e in ['kindle', 'tolino', "kobo", "bookeen"]
if (e in request.headers.get('User-Agent', "").lower())])
if 'content' in kwargs: if 'content' in kwargs:
content = kwargs['content'] content = kwargs['content']
content = isinstance(content, (User, LocalProxy)) and not content.role_anonymous() content = isinstance(content, (User, LocalProxy)) and not content.role_anonymous()
@ -44,12 +47,12 @@ def get_sidebar_config(kwargs=None):
"show_text": _('Show Hot Books'), "config_show": True}) "show_text": _('Show Hot Books'), "config_show": True})
if current_user.role_admin(): if current_user.role_admin():
sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.download_list', sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.download_list',
"id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not g.user.is_anonymous), "id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not current_user.is_anonymous),
"page": "download", "show_text": _('Show Downloaded Books'), "page": "download", "show_text": _('Show Downloaded Books'),
"config_show": content}) "config_show": content})
else: else:
sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.books_list', sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.books_list',
"id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not g.user.is_anonymous), "id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not current_user.is_anonymous),
"page": "download", "show_text": _('Show Downloaded Books'), "page": "download", "show_text": _('Show Downloaded Books'),
"config_show": content}) "config_show": content})
sidebar.append( sidebar.append(
@ -57,66 +60,60 @@ def get_sidebar_config(kwargs=None):
"visibility": constants.SIDEBAR_BEST_RATED, 'public': True, "page": "rated", "visibility": constants.SIDEBAR_BEST_RATED, 'public': True, "page": "rated",
"show_text": _('Show Top Rated Books'), "config_show": True}) "show_text": _('Show Top Rated Books'), "config_show": True})
sidebar.append({"glyph": "glyphicon-eye-open", "text": _('Read Books'), "link": 'web.books_list', "id": "read", sidebar.append({"glyph": "glyphicon-eye-open", "text": _('Read Books'), "link": 'web.books_list', "id": "read",
"visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not g.user.is_anonymous), "visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not current_user.is_anonymous),
"page": "read", "show_text": _('Show read and unread'), "config_show": content}) "page": "read", "show_text": _('Show Read and Unread'), "config_show": content})
sidebar.append( sidebar.append(
{"glyph": "glyphicon-eye-close", "text": _('Unread Books'), "link": 'web.books_list', "id": "unread", {"glyph": "glyphicon-eye-close", "text": _('Unread Books'), "link": 'web.books_list', "id": "unread",
"visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not g.user.is_anonymous), "page": "unread", "visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not current_user.is_anonymous), "page": "unread",
"show_text": _('Show unread'), "config_show": False}) "show_text": _('Show unread'), "config_show": False})
sidebar.append({"glyph": "glyphicon-random", "text": _('Discover'), "link": 'web.books_list', "id": "rand", sidebar.append({"glyph": "glyphicon-random", "text": _('Discover'), "link": 'web.books_list', "id": "rand",
"visibility": constants.SIDEBAR_RANDOM, 'public': True, "page": "discover", "visibility": constants.SIDEBAR_RANDOM, 'public': True, "page": "discover",
"show_text": _('Show Random Books'), "config_show": True}) "show_text": _('Show Random Books'), "config_show": True})
sidebar.append({"glyph": "glyphicon-inbox", "text": _('Categories'), "link": 'web.category_list', "id": "cat", sidebar.append({"glyph": "glyphicon-inbox", "text": _('Categories'), "link": 'web.category_list', "id": "cat",
"visibility": constants.SIDEBAR_CATEGORY, 'public': True, "page": "category", "visibility": constants.SIDEBAR_CATEGORY, 'public': True, "page": "category",
"show_text": _('Show category selection'), "config_show": True}) "show_text": _('Show Category Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-bookmark", "text": _('Series'), "link": 'web.series_list', "id": "serie", sidebar.append({"glyph": "glyphicon-bookmark", "text": _('Series'), "link": 'web.series_list', "id": "serie",
"visibility": constants.SIDEBAR_SERIES, 'public': True, "page": "series", "visibility": constants.SIDEBAR_SERIES, 'public': True, "page": "series",
"show_text": _('Show series selection'), "config_show": True}) "show_text": _('Show Series Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-user", "text": _('Authors'), "link": 'web.author_list', "id": "author", sidebar.append({"glyph": "glyphicon-user", "text": _('Authors'), "link": 'web.author_list', "id": "author",
"visibility": constants.SIDEBAR_AUTHOR, 'public': True, "page": "author", "visibility": constants.SIDEBAR_AUTHOR, 'public': True, "page": "author",
"show_text": _('Show author selection'), "config_show": True}) "show_text": _('Show Author Section'), "config_show": True})
sidebar.append( sidebar.append(
{"glyph": "glyphicon-text-size", "text": _('Publishers'), "link": 'web.publisher_list', "id": "publisher", {"glyph": "glyphicon-text-size", "text": _('Publishers'), "link": 'web.publisher_list', "id": "publisher",
"visibility": constants.SIDEBAR_PUBLISHER, 'public': True, "page": "publisher", "visibility": constants.SIDEBAR_PUBLISHER, 'public': True, "page": "publisher",
"show_text": _('Show publisher selection'), "config_show":True}) "show_text": _('Show Publisher Section'), "config_show":True})
sidebar.append({"glyph": "glyphicon-flag", "text": _('Languages'), "link": 'web.language_overview', "id": "lang", sidebar.append({"glyph": "glyphicon-flag", "text": _('Languages'), "link": 'web.language_overview', "id": "lang",
"visibility": constants.SIDEBAR_LANGUAGE, 'public': (g.user.filter_language() == 'all'), "visibility": constants.SIDEBAR_LANGUAGE, 'public': (current_user.filter_language() == 'all'),
"page": "language", "page": "language",
"show_text": _('Show language selection'), "config_show": True}) "show_text": _('Show Language Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-star-empty", "text": _('Ratings'), "link": 'web.ratings_list', "id": "rate", sidebar.append({"glyph": "glyphicon-star-empty", "text": _('Ratings'), "link": 'web.ratings_list', "id": "rate",
"visibility": constants.SIDEBAR_RATING, 'public': True, "visibility": constants.SIDEBAR_RATING, 'public': True,
"page": "rating", "show_text": _('Show ratings selection'), "config_show": True}) "page": "rating", "show_text": _('Show Ratings Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-file", "text": _('File formats'), "link": 'web.formats_list', "id": "format", sidebar.append({"glyph": "glyphicon-file", "text": _('File formats'), "link": 'web.formats_list', "id": "format",
"visibility": constants.SIDEBAR_FORMAT, 'public': True, "visibility": constants.SIDEBAR_FORMAT, 'public': True,
"page": "format", "show_text": _('Show file formats selection'), "config_show": True}) "page": "format", "show_text": _('Show File Formats Section'), "config_show": True})
sidebar.append( sidebar.append(
{"glyph": "glyphicon-trash", "text": _('Archived Books'), "link": 'web.books_list', "id": "archived", {"glyph": "glyphicon-trash", "text": _('Archived Books'), "link": 'web.books_list', "id": "archived",
"visibility": constants.SIDEBAR_ARCHIVED, 'public': (not g.user.is_anonymous), "page": "archived", "visibility": constants.SIDEBAR_ARCHIVED, 'public': (not current_user.is_anonymous), "page": "archived",
"show_text": _('Show archived books'), "config_show": content}) "show_text": _('Show Archived Books'), "config_show": content})
sidebar.append( if not simple:
{"glyph": "glyphicon-th-list", "text": _('Books List'), "link": 'web.books_table', "id": "list", sidebar.append(
"visibility": constants.SIDEBAR_LIST, 'public': (not g.user.is_anonymous), "page": "list", {"glyph": "glyphicon-th-list", "text": _('Books List'), "link": 'web.books_table', "id": "list",
"show_text": _('Show Books List'), "config_show": content}) "visibility": constants.SIDEBAR_LIST, 'public': (not current_user.is_anonymous), "page": "list",
"show_text": _('Show Books List'), "config_show": content})
g.shelves_access = ub.session.query(ub.Shelf).filter(
or_(ub.Shelf.is_public == 1, ub.Shelf.user_id == current_user.id)).order_by(ub.Shelf.name).all()
return sidebar return sidebar, simple
def get_readbooks_ids():
if not config.config_read_column:
readBooks = ub.session.query(ub.ReadBook).filter(ub.ReadBook.user_id == int(current_user.id))\
.filter(ub.ReadBook.read_status == ub.ReadBook.STATUS_FINISHED).all()
return frozenset([x.book_id for x in readBooks])
else:
try:
readBooks = calibre_db.session.query(db.cc_classes[config.config_read_column])\
.filter(db.cc_classes[config.config_read_column].value == True).all()
return frozenset([x.book for x in readBooks])
except (KeyError, AttributeError):
log.error("Custom Column No.%d is not existing in calibre database", config.config_read_column)
return []
# Returns the template for rendering and includes the instance name # Returns the template for rendering and includes the instance name
def render_title_template(*args, **kwargs): def render_title_template(*args, **kwargs):
sidebar = get_sidebar_config(kwargs) sidebar, simple = get_sidebar_config(kwargs)
return render_template(instance=config.config_calibre_web_title, sidebar=sidebar, try:
accept=constants.EXTENSIONS_UPLOAD, read_book_ids=get_readbooks_ids(), return render_template(instance=config.config_calibre_web_title, sidebar=sidebar, simple=simple,
*args, **kwargs) accept=constants.EXTENSIONS_UPLOAD,
*args, **kwargs)
except PermissionError:
log.error("No permission to access {} file.".format(args[0]))
abort(403)

102
cps/schedule.py Normal file
View File

@ -0,0 +1,102 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2020 mmonkey
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import datetime
from . import config, constants
from .services.background_scheduler import BackgroundScheduler, CronTrigger, use_APScheduler
from .tasks.database import TaskReconnectDatabase
from .tasks.thumbnail import TaskGenerateCoverThumbnails, TaskGenerateSeriesThumbnails, TaskClearCoverThumbnailCache
from .services.worker import WorkerThread
from .tasks.metadata_backup import TaskBackupMetadata
def get_scheduled_tasks(reconnect=True):
tasks = list()
# config.schedule_reconnect or
# Reconnect Calibre database (metadata.db)
if reconnect:
tasks.append([lambda: TaskReconnectDatabase(), 'reconnect', False])
# ToDo make configurable. Generate metadata.opf file for each changed book
if False:
tasks.append([lambda: TaskBackupMetadata("en"), 'backup metadata', False])
# Generate all missing book cover thumbnails
if config.schedule_generate_book_covers:
tasks.append([lambda: TaskClearCoverThumbnailCache(0), 'delete superfluous book covers', True])
tasks.append([lambda: TaskGenerateCoverThumbnails(), 'generate book covers', False])
# Generate all missing series thumbnails
if config.schedule_generate_series_covers:
tasks.append([lambda: TaskGenerateSeriesThumbnails(), 'generate book covers', False])
return tasks
def end_scheduled_tasks():
worker = WorkerThread.get_instance()
for __, __, __, task, __ in worker.tasks:
if task.scheduled and task.is_cancellable:
worker.end_task(task.id)
def register_scheduled_tasks(reconnect=True):
scheduler = BackgroundScheduler()
if scheduler:
# Remove all existing jobs
scheduler.remove_all_jobs()
start = config.schedule_start_time
duration = config.schedule_duration
# Register scheduled tasks
scheduler.schedule_tasks(tasks=get_scheduled_tasks(reconnect), trigger=CronTrigger(hour=start))
end_time = calclulate_end_time(start, duration)
scheduler.schedule(func=end_scheduled_tasks, trigger=CronTrigger(hour=end_time.hour, minute=end_time.minute),
name="end scheduled task")
# Kick-off tasks, if they should currently be running
if should_task_be_running(start, duration):
scheduler.schedule_tasks_immediately(tasks=get_scheduled_tasks(reconnect))
def register_startup_tasks():
scheduler = BackgroundScheduler()
if scheduler:
start = config.schedule_start_time
duration = config.schedule_duration
# Run scheduled tasks immediately for development and testing
# Ignore tasks that should currently be running, as these will be added when registering scheduled tasks
if constants.APP_MODE in ['development', 'test'] and not should_task_be_running(start, duration):
scheduler.schedule_tasks_immediately(tasks=get_scheduled_tasks(False))
def should_task_be_running(start, duration):
now = datetime.datetime.now()
start_time = datetime.datetime.now().replace(hour=start, minute=0, second=0, microsecond=0)
end_time = start_time + datetime.timedelta(hours=duration // 60, minutes=duration % 60)
return start_time < now < end_time
def calclulate_end_time(start, duration):
start_time = datetime.datetime.now().replace(hour=start, minute=0)
return start_time + datetime.timedelta(hours=duration // 60, minutes=duration % 60)

397
cps/search.py Normal file
View File

@ -0,0 +1,397 @@
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import json
from datetime import datetime
from flask import Blueprint, request, redirect, url_for, flash
from flask import session as flask_session
from flask_login import current_user
from flask_babel import format_date
from flask_babel import gettext as _
from sqlalchemy.sql.expression import func, not_, and_, or_, text, true
from sqlalchemy.sql.functions import coalesce
from . import logger, db, calibre_db, config, ub
from .usermanagement import login_required_if_no_ano
from .render_template import render_title_template
from .pagination import Pagination
search = Blueprint('search', __name__)
log = logger.create()
@search.route("/search", methods=["POST"])
@login_required_if_no_ano
def simple_search():
term = dict(request.form).get("query")
if term:
flask_session['query'] = json.dumps(term.strip())
return redirect(url_for('web.books_list', data="search", sort_param='stored', query="")) # term.strip()
else:
return render_title_template('search.html',
searchterm="",
result_count=0,
title=_("Search"),
page="search")
@search.route("/advsearch", methods=['POST'])
@login_required_if_no_ano
def advanced_search():
values = dict(request.form)
params = ['include_tag', 'exclude_tag', 'include_serie', 'exclude_serie', 'include_shelf', 'exclude_shelf',
'include_language', 'exclude_language', 'include_extension', 'exclude_extension']
for param in params:
values[param] = list(request.form.getlist(param))
flask_session['query'] = json.dumps(values)
return redirect(url_for('web.books_list', data="advsearch", sort_param='stored', query=""))
@search.route("/advsearch", methods=['GET'])
@login_required_if_no_ano
def advanced_search_form():
# Build custom columns names
cc = calibre_db.get_cc_columns(config, filter_config_custom_read=True)
return render_prepare_search_form(cc)
def adv_search_custom_columns(cc, term, q):
for c in cc:
if c.datatype == "datetime":
custom_start = term.get('custom_column_' + str(c.id) + '_start')
custom_end = term.get('custom_column_' + str(c.id) + '_end')
if custom_start:
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
func.datetime(db.cc_classes[c.id].value) >= func.datetime(custom_start)))
if custom_end:
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
func.datetime(db.cc_classes[c.id].value) <= func.datetime(custom_end)))
else:
custom_query = term.get('custom_column_' + str(c.id))
if custom_query != '' and custom_query is not None:
if c.datatype == 'bool':
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
db.cc_classes[c.id].value == (custom_query == "True")))
elif c.datatype == 'int' or c.datatype == 'float':
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
db.cc_classes[c.id].value == custom_query))
elif c.datatype == 'rating':
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
db.cc_classes[c.id].value == int(float(custom_query) * 2)))
else:
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
func.lower(db.cc_classes[c.id].value).ilike("%" + custom_query + "%")))
return q
def adv_search_language(q, include_languages_inputs, exclude_languages_inputs):
if current_user.filter_language() != "all":
q = q.filter(db.Books.languages.any(db.Languages.lang_code == current_user.filter_language()))
else:
for language in include_languages_inputs:
q = q.filter(db.Books.languages.any(db.Languages.id == language))
for language in exclude_languages_inputs:
q = q.filter(not_(db.Books.series.any(db.Languages.id == language)))
return q
def adv_search_ratings(q, rating_high, rating_low):
if rating_high:
rating_high = int(rating_high) * 2
q = q.filter(db.Books.ratings.any(db.Ratings.rating <= rating_high))
if rating_low:
rating_low = int(rating_low) * 2
q = q.filter(db.Books.ratings.any(db.Ratings.rating >= rating_low))
return q
def adv_search_read_status(read_status):
if not config.config_read_column:
if read_status == "True":
db_filter = and_(ub.ReadBook.user_id == int(current_user.id),
ub.ReadBook.read_status == ub.ReadBook.STATUS_FINISHED)
else:
db_filter = coalesce(ub.ReadBook.read_status, 0) != ub.ReadBook.STATUS_FINISHED
else:
try:
if read_status == "True":
db_filter = db.cc_classes[config.config_read_column].value == True
else:
db_filter = coalesce(db.cc_classes[config.config_read_column].value, False) != True
except (KeyError, AttributeError, IndexError):
log.error("Custom Column No.{} does not exist in calibre database".format(config.config_read_column))
flash(_("Custom Column No.%(column)d does not exist in calibre database",
column=config.config_read_column),
category="error")
return true()
return db_filter
def adv_search_extension(q, include_extension_inputs, exclude_extension_inputs):
for extension in include_extension_inputs:
q = q.filter(db.Books.data.any(db.Data.format == extension))
for extension in exclude_extension_inputs:
q = q.filter(not_(db.Books.data.any(db.Data.format == extension)))
return q
def adv_search_tag(q, include_tag_inputs, exclude_tag_inputs):
for tag in include_tag_inputs:
q = q.filter(db.Books.tags.any(db.Tags.id == tag))
for tag in exclude_tag_inputs:
q = q.filter(not_(db.Books.tags.any(db.Tags.id == tag)))
return q
def adv_search_serie(q, include_series_inputs, exclude_series_inputs):
for serie in include_series_inputs:
q = q.filter(db.Books.series.any(db.Series.id == serie))
for serie in exclude_series_inputs:
q = q.filter(not_(db.Books.series.any(db.Series.id == serie)))
return q
def adv_search_shelf(q, include_shelf_inputs, exclude_shelf_inputs):
q = q.outerjoin(ub.BookShelf, db.Books.id == ub.BookShelf.book_id)\
.filter(or_(ub.BookShelf.shelf == None, ub.BookShelf.shelf.notin_(exclude_shelf_inputs)))
if len(include_shelf_inputs) > 0:
q = q.filter(ub.BookShelf.shelf.in_(include_shelf_inputs))
return q
def extend_search_term(searchterm,
author_name,
book_title,
publisher,
pub_start,
pub_end,
tags,
rating_high,
rating_low,
read_status,
):
searchterm.extend((author_name.replace('|', ','), book_title, publisher))
if pub_start:
try:
searchterm.extend([_("Published after ") +
format_date(datetime.strptime(pub_start, "%Y-%m-%d"),
format='medium')])
except ValueError:
pub_start = ""
if pub_end:
try:
searchterm.extend([_("Published before ") +
format_date(datetime.strptime(pub_end, "%Y-%m-%d"),
format='medium')])
except ValueError:
pub_end = ""
elements = {'tag': db.Tags, 'serie':db.Series, 'shelf':ub.Shelf}
for key, db_element in elements.items():
tag_names = calibre_db.session.query(db_element).filter(db_element.id.in_(tags['include_' + key])).all()
searchterm.extend(tag.name for tag in tag_names)
tag_names = calibre_db.session.query(db_element).filter(db_element.id.in_(tags['exclude_' + key])).all()
searchterm.extend(tag.name for tag in tag_names)
language_names = calibre_db.session.query(db.Languages). \
filter(db.Languages.id.in_(tags['include_language'])).all()
if language_names:
language_names = calibre_db.speaking_language(language_names)
searchterm.extend(language.name for language in language_names)
language_names = calibre_db.session.query(db.Languages). \
filter(db.Languages.id.in_(tags['exclude_language'])).all()
if language_names:
language_names = calibre_db.speaking_language(language_names)
searchterm.extend(language.name for language in language_names)
if rating_high:
searchterm.extend([_("Rating <= %(rating)s", rating=rating_high)])
if rating_low:
searchterm.extend([_("Rating >= %(rating)s", rating=rating_low)])
if read_status:
searchterm.extend([_("Read Status = %(status)s", status=read_status)])
searchterm.extend(ext for ext in tags['include_extension'])
searchterm.extend(ext for ext in tags['exclude_extension'])
# handle custom columns
searchterm = " + ".join(filter(None, searchterm))
return searchterm, pub_start, pub_end
def render_adv_search_results(term, offset=None, order=None, limit=None):
sort = order[0] if order else [db.Books.sort]
pagination = None
cc = calibre_db.get_cc_columns(config, filter_config_custom_read=True)
calibre_db.session.connection().connection.connection.create_function("lower", 1, db.lcase)
query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
q = query.outerjoin(db.books_series_link, db.Books.id == db.books_series_link.c.book)\
.outerjoin(db.Series)\
.filter(calibre_db.common_filters(True))
# parse multi selects to a complete dict
tags = dict()
elements = ['tag', 'serie', 'shelf', 'language', 'extension']
for element in elements:
tags['include_' + element] = term.get('include_' + element)
tags['exclude_' + element] = term.get('exclude_' + element)
author_name = term.get("author_name")
book_title = term.get("book_title")
publisher = term.get("publisher")
pub_start = term.get("publishstart")
pub_end = term.get("publishend")
rating_low = term.get("ratinghigh")
rating_high = term.get("ratinglow")
description = term.get("comment")
read_status = term.get("read_status")
if author_name:
author_name = author_name.strip().lower().replace(',', '|')
if book_title:
book_title = book_title.strip().lower()
if publisher:
publisher = publisher.strip().lower()
search_term = []
cc_present = False
for c in cc:
if c.datatype == "datetime":
column_start = term.get('custom_column_' + str(c.id) + '_start')
column_end = term.get('custom_column_' + str(c.id) + '_end')
if column_start:
search_term.extend(["{} >= {}".format(c.name,
format_date(datetime.strptime(column_start, "%Y-%m-%d").date(),
format='medium')
)])
cc_present = True
if column_end:
search_term.extend(["{} <= {}".format(c.name,
format_date(datetime.strptime(column_end, "%Y-%m-%d").date(),
format='medium')
)])
cc_present = True
elif term.get('custom_column_' + str(c.id)):
search_term.extend([("{}: {}".format(c.name, term.get('custom_column_' + str(c.id))))])
cc_present = True
if any(tags.values()) or author_name or book_title or publisher or pub_start or pub_end or rating_low \
or rating_high or description or cc_present or read_status:
search_term, pub_start, pub_end = extend_search_term(search_term,
author_name,
book_title,
publisher,
pub_start,
pub_end,
tags,
rating_high,
rating_low,
read_status)
if author_name:
q = q.filter(db.Books.authors.any(func.lower(db.Authors.name).ilike("%" + author_name + "%")))
if book_title:
q = q.filter(func.lower(db.Books.title).ilike("%" + book_title + "%"))
if pub_start:
q = q.filter(func.datetime(db.Books.pubdate) > func.datetime(pub_start))
if pub_end:
q = q.filter(func.datetime(db.Books.pubdate) < func.datetime(pub_end))
q = q.filter(adv_search_read_status(read_status))
if publisher:
q = q.filter(db.Books.publishers.any(func.lower(db.Publishers.name).ilike("%" + publisher + "%")))
q = adv_search_tag(q, tags['include_tag'], tags['exclude_tag'])
q = adv_search_serie(q, tags['include_serie'], tags['exclude_serie'])
q = adv_search_shelf(q, tags['include_shelf'], tags['exclude_shelf'])
q = adv_search_extension(q, tags['include_extension'], tags['exclude_extension'])
q = adv_search_language(q, tags['include_language'], tags['exclude_language'])
q = adv_search_ratings(q, rating_high, rating_low)
if description:
q = q.filter(db.Books.comments.any(func.lower(db.Comments.text).ilike("%" + description + "%")))
# search custom columns
try:
q = adv_search_custom_columns(cc, term, q)
except AttributeError as ex:
log.debug_or_exception(ex)
flash(_("Error on search for custom columns, please restart Calibre-Web"), category="error")
q = q.order_by(*sort).all()
flask_session['query'] = json.dumps(term)
ub.store_combo_ids(q)
result_count = len(q)
if offset is not None and limit is not None:
offset = int(offset)
limit_all = offset + int(limit)
pagination = Pagination((offset / (int(limit)) + 1), limit, result_count)
else:
offset = 0
limit_all = result_count
entries = calibre_db.order_authors(q[offset:limit_all], list_return=True, combined=True)
return render_title_template('search.html',
adv_searchterm=search_term,
pagination=pagination,
entries=entries,
result_count=result_count,
title=_("Advanced Search"), page="advsearch",
order=order[1])
def render_prepare_search_form(cc):
# prepare data for search-form
tags = calibre_db.session.query(db.Tags)\
.join(db.books_tags_link)\
.join(db.Books)\
.filter(calibre_db.common_filters()) \
.group_by(text('books_tags_link.tag'))\
.order_by(db.Tags.name).all()
series = calibre_db.session.query(db.Series)\
.join(db.books_series_link)\
.join(db.Books)\
.filter(calibre_db.common_filters()) \
.group_by(text('books_series_link.series'))\
.order_by(db.Series.name)\
.filter(calibre_db.common_filters()).all()
shelves = ub.session.query(ub.Shelf)\
.filter(or_(ub.Shelf.is_public == 1, ub.Shelf.user_id == int(current_user.id)))\
.order_by(ub.Shelf.name).all()
extensions = calibre_db.session.query(db.Data)\
.join(db.Books)\
.filter(calibre_db.common_filters()) \
.group_by(db.Data.format)\
.order_by(db.Data.format).all()
if current_user.filter_language() == "all":
languages = calibre_db.speaking_language()
else:
languages = None
return render_title_template('search_form.html', tags=tags, languages=languages, extensions=extensions,
series=series,shelves=shelves, title=_("Advanced Search"), cc=cc, page="advsearch")
def render_search_results(term, offset=None, order=None, limit=None):
join = db.books_series_link, db.Books.id == db.books_series_link.c.book, db.Series
entries, result_count, pagination = calibre_db.get_search_results(term,
config,
offset,
order,
limit,
*join)
return render_title_template('search.html',
searchterm=term,
pagination=pagination,
query=term,
adv_searchterm=term,
entries=entries,
result_count=result_count,
title=_("Search"),
page="search",
order=order[1])

110
cps/search_metadata.py
View File

@ -16,69 +16,91 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
import json
import importlib
import sys
import inspect
import datetime
import concurrent.futures import concurrent.futures
import importlib
import inspect
import json
import os
import sys
from flask import Blueprint, request, Response, url_for from flask import Blueprint, Response, request, url_for
from flask_login import current_user from flask_login import current_user
from flask_login import login_required from flask_login import login_required
from flask_babel import get_locale
from sqlalchemy.exc import InvalidRequestError, OperationalError
from sqlalchemy.orm.attributes import flag_modified from sqlalchemy.orm.attributes import flag_modified
from sqlalchemy.exc import OperationalError, InvalidRequestError
from . import constants, logger, ub
from cps.services.Metadata import Metadata from cps.services.Metadata import Metadata
from . import constants, logger, ub, web_server
# current_milli_time = lambda: int(round(time() * 1000))
meta = Blueprint('metadata', __name__) meta = Blueprint("metadata", __name__)
log = logger.create() log = logger.create()
try:
from dataclasses import asdict
except ImportError:
log.info('*** "dataclasses" is needed for calibre-web to run. Please install it using pip: "pip install dataclasses" ***')
print('*** "dataclasses" is needed for calibre-web to run. Please install it using pip: "pip install dataclasses" ***')
web_server.stop(True)
sys.exit(6)
new_list = list() new_list = list()
meta_dir = os.path.join(constants.BASE_DIR, "cps", "metadata_provider") meta_dir = os.path.join(constants.BASE_DIR, "cps", "metadata_provider")
modules = os.listdir(os.path.join(constants.BASE_DIR, "cps", "metadata_provider")) modules = os.listdir(os.path.join(constants.BASE_DIR, "cps", "metadata_provider"))
for f in modules: for f in modules:
if os.path.isfile(os.path.join(meta_dir, f)) and not f.endswith('__init__.py'): if os.path.isfile(os.path.join(meta_dir, f)) and not f.endswith("__init__.py"):
a = os.path.basename(f)[:-3] a = os.path.basename(f)[:-3]
try: try:
importlib.import_module("cps.metadata_provider." + a) importlib.import_module("cps.metadata_provider." + a)
new_list.append(a) new_list.append(a)
except ImportError: except (IndentationError, SyntaxError) as e:
log.error("Import error for metadata source: {}".format(a)) log.error("Syntax error for metadata source: {} - {}".format(a, e))
pass except ImportError as e:
log.debug("Import error for metadata source: {} - {}".format(a, e))
def list_classes(provider_list): def list_classes(provider_list):
classes = list() classes = list()
for element in provider_list: for element in provider_list:
for name, obj in inspect.getmembers(sys.modules["cps.metadata_provider." + element]): for name, obj in inspect.getmembers(
if inspect.isclass(obj) and name != "Metadata" and issubclass(obj, Metadata): sys.modules["cps.metadata_provider." + element]
):
if (
inspect.isclass(obj)
and name != "Metadata"
and issubclass(obj, Metadata)
):
classes.append(obj()) classes.append(obj())
return classes return classes
cl = list_classes(new_list) cl = list_classes(new_list)
@meta.route("/metadata/provider") @meta.route("/metadata/provider")
@login_required @login_required
def metadata_provider(): def metadata_provider():
active = current_user.view_settings.get('metadata', {}) active = current_user.view_settings.get("metadata", {})
provider = list() provider = list()
for c in cl: for c in cl:
ac = active.get(c.__id__, True) ac = active.get(c.__id__, True)
provider.append({"name": c.__name__, "active": ac, "initial": ac, "id": c.__id__}) provider.append(
return Response(json.dumps(provider), mimetype='application/json') {"name": c.__name__, "active": ac, "initial": ac, "id": c.__id__}
)
return Response(json.dumps(provider), mimetype="application/json")
@meta.route("/metadata/provider", methods=['POST'])
@meta.route("/metadata/provider/<prov_name>", methods=['POST']) @meta.route("/metadata/provider", methods=["POST"])
@meta.route("/metadata/provider/<prov_name>", methods=["POST"])
@login_required @login_required
def metadata_change_active_provider(prov_name): def metadata_change_active_provider(prov_name):
new_state = request.get_json() new_state = request.get_json()
active = current_user.view_settings.get('metadata', {}) active = current_user.view_settings.get("metadata", {})
active[new_state['id']] = new_state['value'] active[new_state["id"]] = new_state["value"]
current_user.view_settings['metadata'] = active current_user.view_settings["metadata"] = active
try: try:
try: try:
flag_modified(current_user, "view_settings") flag_modified(current_user, "view_settings")
@ -89,29 +111,33 @@ def metadata_change_active_provider(prov_name):
log.error("Invalid request received: {}".format(request)) log.error("Invalid request received: {}".format(request))
return "Invalid request", 400 return "Invalid request", 400
if "initial" in new_state and prov_name: if "initial" in new_state and prov_name:
for c in cl: data = []
if c.__id__ == prov_name: provider = next((c for c in cl if c.__id__ == prov_name), None)
data = c.search(new_state.get('query', "")) if provider is not None:
break data = provider.search(new_state.get("query", ""))
return Response(json.dumps(data), mimetype='application/json') return Response(
json.dumps([asdict(x) for x in data]), mimetype="application/json"
)
return "" return ""
@meta.route("/metadata/search", methods=['POST'])
@meta.route("/metadata/search", methods=["POST"])
@login_required @login_required
def metadata_search(): def metadata_search():
query = request.form.to_dict().get('query') query = request.form.to_dict().get("query")
data = list() data = list()
active = current_user.view_settings.get('metadata', {}) active = current_user.view_settings.get("metadata", {})
locale = get_locale()
if query: if query:
static_cover = url_for('static', filename='generic_cover.jpg') static_cover = url_for("static", filename="generic_cover.jpg")
# start = current_milli_time()
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
meta = {executor.submit(c.search, query, static_cover): c for c in cl if active.get(c.__id__, True)} meta = {
executor.submit(c.search, query, static_cover, locale): c
for c in cl
if active.get(c.__id__, True)
}
for future in concurrent.futures.as_completed(meta): for future in concurrent.futures.as_completed(meta):
data.extend(future.result()) data.extend([asdict(x) for x in future.result() if x])
return Response(json.dumps(data), mimetype='application/json') # log.info({'Time elapsed {}'.format(current_milli_time()-start)})
return Response(json.dumps(data), mimetype="application/json")

19
cps/server.py
View File

@ -22,9 +22,11 @@ import errno
import signal import signal
import socket import socket
import subprocess # nosec import subprocess # nosec
from .services.background_scheduler import BackgroundScheduler
try: try:
from gevent.pywsgi import WSGIServer from gevent.pywsgi import WSGIServer
from .gevent_wsgi import MyWSGIHandler
from gevent.pool import Pool from gevent.pool import Pool
from gevent import __version__ as _version from gevent import __version__ as _version
from greenlet import GreenletExit from greenlet import GreenletExit
@ -32,7 +34,7 @@ try:
VERSION = 'Gevent ' + _version VERSION = 'Gevent ' + _version
_GEVENT = True _GEVENT = True
except ImportError: except ImportError:
from tornado.wsgi import WSGIContainer from .tornado_wsgi import MyWSGIContainer
from tornado.httpserver import HTTPServer from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop from tornado.ioloop import IOLoop
from tornado import version as _version from tornado import version as _version
@ -202,7 +204,9 @@ class WebServer(object):
if output is None: if output is None:
output = _readable_listen_address(self.listen_address, self.listen_port) output = _readable_listen_address(self.listen_address, self.listen_port)
log.info('Starting Gevent server on %s', output) log.info('Starting Gevent server on %s', output)
self.wsgiserver = WSGIServer(sock, self.app, log=self.access_logger, spawn=Pool(), **ssl_args) self.wsgiserver = WSGIServer(sock, self.app, log=self.access_logger, handler_class=MyWSGIHandler,
error_log=log,
spawn=Pool(), **ssl_args)
if ssl_args: if ssl_args:
wrap_socket = self.wsgiserver.wrap_socket wrap_socket = self.wsgiserver.wrap_socket
def my_wrap_socket(*args, **kwargs): def my_wrap_socket(*args, **kwargs):
@ -225,8 +229,8 @@ class WebServer(object):
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
log.info('Starting Tornado server on %s', _readable_listen_address(self.listen_address, self.listen_port)) log.info('Starting Tornado server on %s', _readable_listen_address(self.listen_address, self.listen_port))
# Max Buffersize set to 200MB ) # Max Buffersize set to 200MB
http_server = HTTPServer(WSGIContainer(self.app), http_server = HTTPServer(MyWSGIContainer(self.app),
max_buffer_size=209700000, max_buffer_size=209700000,
ssl_options=self.ssl_args) ssl_options=self.ssl_args)
http_server.listen(self.listen_port, self.listen_address) http_server.listen(self.listen_port, self.listen_address)
@ -262,6 +266,12 @@ class WebServer(object):
subprocess.call(args, close_fds=True) # nosec subprocess.call(args, close_fds=True) # nosec
return True return True
@staticmethod
def shutdown_scheduler():
scheduler = BackgroundScheduler()
if scheduler:
scheduler.scheduler.shutdown()
def _killServer(self, __, ___): def _killServer(self, __, ___):
self.stop() self.stop()
@ -270,6 +280,7 @@ class WebServer(object):
updater_thread.stop() updater_thread.stop()
log.info("webserver stop (restart=%s)", restart) log.info("webserver stop (restart=%s)", restart)
self.shutdown_scheduler()
self.restart = restart self.restart = restart
if self.wsgiserver: if self.wsgiserver:
if _GEVENT: if _GEVENT:

86
cps/services/Metadata.py
View File

@ -15,13 +15,97 @@
# #
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import abc
import dataclasses
import os
import re
from typing import Dict, Generator, List, Optional, Union
from cps import constants
class Metadata(): @dataclasses.dataclass
class MetaSourceInfo:
id: str
description: str
link: str
@dataclasses.dataclass
class MetaRecord:
id: Union[str, int]
title: str
authors: List[str]
url: str
source: MetaSourceInfo
cover: str = os.path.join(constants.STATIC_DIR, 'generic_cover.jpg')
description: Optional[str] = ""
series: Optional[str] = None
series_index: Optional[Union[int, float]] = 0
identifiers: Dict[str, Union[str, int]] = dataclasses.field(default_factory=dict)
publisher: Optional[str] = None
publishedDate: Optional[str] = None
rating: Optional[int] = 0
languages: Optional[List[str]] = dataclasses.field(default_factory=list)
tags: Optional[List[str]] = dataclasses.field(default_factory=list)
class Metadata:
__name__ = "Generic" __name__ = "Generic"
__id__ = "generic"
def __init__(self): def __init__(self):
self.active = True self.active = True
def set_status(self, state): def set_status(self, state):
self.active = state self.active = state
@abc.abstractmethod
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
pass
@staticmethod
def get_title_tokens(
title: str, strip_joiners: bool = True
) -> Generator[str, None, None]:
"""
Taken from calibre source code
It's a simplified (cut out what is unnecessary) version of
https://github.com/kovidgoyal/calibre/blob/99d85b97918625d172227c8ffb7e0c71794966c0/
src/calibre/ebooks/metadata/sources/base.py#L363-L367
(src/calibre/ebooks/metadata/sources/base.py - lines 363-398)
"""
title_patterns = [
(re.compile(pat, re.IGNORECASE), repl)
for pat, repl in [
# Remove things like: (2010) (Omnibus) etc.
(
r"(?i)[({\[](\d{4}|omnibus|anthology|hardcover|"
r"audiobook|audio\scd|paperback|turtleback|"
r"mass\s*market|edition|ed\.)[\])}]",
"",
),
# Remove any strings that contain the substring edition inside
# parentheses
(r"(?i)[({\[].*?(edition|ed.).*?[\]})]", ""),
# Remove commas used a separators in numbers
(r"(\d+),(\d+)", r"\1\2"),
# Remove hyphens only if they have whitespace before them
(r"(\s-)", " "),
# Replace other special chars with a space
(r"""[:,;!@$%^&*(){}.`~"\s\[\]/]《》「」“”""", " "),
]
]
for pat, repl in title_patterns:
title = pat.sub(repl, title)
tokens = title.split()
for token in tokens:
token = token.strip().strip('"').strip("'")
if token and (
not strip_joiners or token.lower() not in ("a", "and", "the", "&")
):
yield token

19
cps/services/SyncToken.py
View File

@ -21,11 +21,8 @@ import sys
from base64 import b64decode, b64encode from base64 import b64decode, b64encode
from jsonschema import validate, exceptions, __version__ from jsonschema import validate, exceptions, __version__
from datetime import datetime from datetime import datetime
try:
# pylint: disable=unused-import from urllib.parse import unquote
from urllib import unquote
except ImportError:
from urllib.parse import unquote
from flask import json from flask import json
from .. import logger from .. import logger
@ -138,12 +135,9 @@ class SyncToken:
archive_last_modified = get_datetime_from_json(data_json, "archive_last_modified") archive_last_modified = get_datetime_from_json(data_json, "archive_last_modified")
reading_state_last_modified = get_datetime_from_json(data_json, "reading_state_last_modified") reading_state_last_modified = get_datetime_from_json(data_json, "reading_state_last_modified")
tags_last_modified = get_datetime_from_json(data_json, "tags_last_modified") tags_last_modified = get_datetime_from_json(data_json, "tags_last_modified")
# books_last_id = data_json["books_last_id"]
except TypeError: except TypeError:
log.error("SyncToken timestamps don't parse to a datetime.") log.error("SyncToken timestamps don't parse to a datetime.")
return SyncToken(raw_kobo_store_token=raw_kobo_store_token) return SyncToken(raw_kobo_store_token=raw_kobo_store_token)
#except KeyError:
# books_last_id = -1
return SyncToken( return SyncToken(
raw_kobo_store_token=raw_kobo_store_token, raw_kobo_store_token=raw_kobo_store_token,
@ -152,7 +146,6 @@ class SyncToken:
archive_last_modified=archive_last_modified, archive_last_modified=archive_last_modified,
reading_state_last_modified=reading_state_last_modified, reading_state_last_modified=reading_state_last_modified,
tags_last_modified=tags_last_modified, tags_last_modified=tags_last_modified,
#books_last_id=books_last_id
) )
def set_kobo_store_header(self, store_headers): def set_kobo_store_header(self, store_headers):
@ -176,16 +169,14 @@ class SyncToken:
"archive_last_modified": to_epoch_timestamp(self.archive_last_modified), "archive_last_modified": to_epoch_timestamp(self.archive_last_modified),
"reading_state_last_modified": to_epoch_timestamp(self.reading_state_last_modified), "reading_state_last_modified": to_epoch_timestamp(self.reading_state_last_modified),
"tags_last_modified": to_epoch_timestamp(self.tags_last_modified), "tags_last_modified": to_epoch_timestamp(self.tags_last_modified),
#"books_last_id":self.books_last_id
}, },
} }
return b64encode_json(token) return b64encode_json(token)
def __str__(self): def __str__(self):
return "{},{},{},{},{},{}".format(self.raw_kobo_store_token, return "{},{},{},{},{},{}".format(self.books_last_created,
self.books_last_created,
self.books_last_modified, self.books_last_modified,
self.archive_last_modified, self.archive_last_modified,
self.reading_state_last_modified, self.reading_state_last_modified,
self.tags_last_modified) self.tags_last_modified,
#self.books_last_id) self.raw_kobo_store_token)

5
cps/services/__init__.py
View File

@ -18,11 +18,10 @@
from .. import logger from .. import logger
log = logger.create() log = logger.create()
try:
try: from . import goodreads_support from . import goodreads_support
except ImportError as err: except ImportError as err:
log.debug("Cannot import goodreads, showing authors-metadata will not work: %s", err) log.debug("Cannot import goodreads, showing authors-metadata will not work: %s", err)
goodreads_support = None goodreads_support = None

84
cps/services/background_scheduler.py Normal file
View File

@ -0,0 +1,84 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2020 mmonkey
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import atexit
from .. import logger
from .worker import WorkerThread
try:
from apscheduler.schedulers.background import BackgroundScheduler as BScheduler
from apscheduler.triggers.cron import CronTrigger
from apscheduler.triggers.date import DateTrigger
use_APScheduler = True
except (ImportError, RuntimeError) as e:
use_APScheduler = False
log = logger.create()
log.info('APScheduler not found. Unable to schedule tasks.')
class BackgroundScheduler:
_instance = None
def __new__(cls):
if not use_APScheduler:
return False
if cls._instance is None:
cls._instance = super(BackgroundScheduler, cls).__new__(cls)
cls.log = logger.create()
cls.scheduler = BScheduler()
cls.scheduler.start()
return cls._instance
def schedule(self, func, trigger, name=None):
if use_APScheduler:
return self.scheduler.add_job(func=func, trigger=trigger, name=name)
# Expects a lambda expression for the task
def schedule_task(self, task, user=None, name=None, hidden=False, trigger=None):
if use_APScheduler:
def scheduled_task():
worker_task = task()
worker_task.scheduled = True
WorkerThread.add(user, worker_task, hidden=hidden)
return self.schedule(func=scheduled_task, trigger=trigger, name=name)
# Expects a list of lambda expressions for the tasks
def schedule_tasks(self, tasks, user=None, trigger=None):
if use_APScheduler:
for task in tasks:
self.schedule_task(task[0], user=user, trigger=trigger, name=task[1], hidden=task[2])
# Expects a lambda expression for the task
def schedule_task_immediately(self, task, user=None, name=None, hidden=False):
if use_APScheduler:
def immediate_task():
WorkerThread.add(user, task(), hidden)
return self.schedule(func=immediate_task, trigger=DateTrigger(), name=name)
# Expects a list of lambda expressions for the tasks
def schedule_tasks_immediately(self, tasks, user=None):
if use_APScheduler:
for task in tasks:
self.schedule_task_immediately(task[0], user, name="immediately " + task[1], hidden=task[2])
# Remove all jobs
def remove_all_jobs(self):
self.scheduler.remove_all_jobs()

6
cps/services/gmail.py
View File

@ -25,7 +25,7 @@ from google.oauth2.credentials import Credentials
from datetime import datetime from datetime import datetime
import base64 import base64
from flask_babel import gettext as _ from flask_babel import gettext as _
from ..constants import BASE_DIR from ..constants import CONFIG_DIR
from .. import logger from .. import logger
@ -53,11 +53,11 @@ def setup_gmail(token):
if creds and creds.expired and creds.refresh_token: if creds and creds.expired and creds.refresh_token:
creds.refresh(Request()) creds.refresh(Request())
else: else:
cred_file = os.path.join(BASE_DIR, 'gmail.json') cred_file = os.path.join(CONFIG_DIR, 'gmail.json')
if not os.path.exists(cred_file): if not os.path.exists(cred_file):
raise Exception(_("Found no valid gmail.json file with OAuth information")) raise Exception(_("Found no valid gmail.json file with OAuth information"))
flow = InstalledAppFlow.from_client_secrets_file( flow = InstalledAppFlow.from_client_secrets_file(
os.path.join(BASE_DIR, 'gmail.json'), SCOPES) os.path.join(CONFIG_DIR, 'gmail.json'), SCOPES)
creds = flow.run_local_server(port=0) creds = flow.run_local_server(port=0)
user_info = get_user_info(creds) user_info = get_user_info(creds)
return { return {

71
cps/services/worker.py
View File

@ -37,11 +37,13 @@ STAT_WAITING = 0
STAT_FAIL = 1 STAT_FAIL = 1
STAT_STARTED = 2 STAT_STARTED = 2
STAT_FINISH_SUCCESS = 3 STAT_FINISH_SUCCESS = 3
STAT_ENDED = 4
STAT_CANCELLED = 5
# Only retain this many tasks in dequeued list # Only retain this many tasks in dequeued list
TASK_CLEANUP_TRIGGER = 20 TASK_CLEANUP_TRIGGER = 20
QueuedTask = namedtuple('QueuedTask', 'num, user, added, task') QueuedTask = namedtuple('QueuedTask', 'num, user, added, task, hidden')
def _get_main_thread(): def _get_main_thread():
@ -51,7 +53,6 @@ def _get_main_thread():
raise Exception("main thread not found?!") raise Exception("main thread not found?!")
class ImprovedQueue(queue.Queue): class ImprovedQueue(queue.Queue):
def to_list(self): def to_list(self):
""" """
@ -61,12 +62,13 @@ class ImprovedQueue(queue.Queue):
with self.mutex: with self.mutex:
return list(self.queue) return list(self.queue)
# Class for all worker tasks in the background # Class for all worker tasks in the background
class WorkerThread(threading.Thread): class WorkerThread(threading.Thread):
_instance = None _instance = None
@classmethod @classmethod
def getInstance(cls): def get_instance(cls):
if cls._instance is None: if cls._instance is None:
cls._instance = WorkerThread() cls._instance = WorkerThread()
return cls._instance return cls._instance
@ -82,15 +84,17 @@ class WorkerThread(threading.Thread):
self.start() self.start()
@classmethod @classmethod
def add(cls, user, task): def add(cls, user, task, hidden=False):
ins = cls.getInstance() ins = cls.get_instance()
ins.num += 1 ins.num += 1
log.debug("Add Task for user: {}: {}".format(user, task)) username = user if user is not None else 'System'
log.debug("Add Task for user: {} - {}".format(username, task))
ins.queue.put(QueuedTask( ins.queue.put(QueuedTask(
num=ins.num, num=ins.num,
user=user, user=username,
added=datetime.now(), added=datetime.now(),
task=task, task=task,
hidden=hidden
)) ))
@property @property
@ -111,10 +115,10 @@ class WorkerThread(threading.Thread):
if delta > TASK_CLEANUP_TRIGGER: if delta > TASK_CLEANUP_TRIGGER:
ret = alive ret = alive
else: else:
# otherwise, lop off the oldest dead tasks until we hit the target trigger # otherwise, loop off the oldest dead tasks until we hit the target trigger
ret = sorted(dead, key=lambda x: x.task.end_time)[-TASK_CLEANUP_TRIGGER:] + alive ret = sorted(dead, key=lambda y: y.task.end_time)[-TASK_CLEANUP_TRIGGER:] + alive
self.dequeued = sorted(ret, key=lambda x: x.num) self.dequeued = sorted(ret, key=lambda y: y.num)
# Main thread loop starting the different tasks # Main thread loop starting the different tasks
def run(self): def run(self):
@ -141,11 +145,21 @@ class WorkerThread(threading.Thread):
# sometimes tasks (like Upload) don't actually have work to do and are created as already finished # sometimes tasks (like Upload) don't actually have work to do and are created as already finished
if item.task.stat is STAT_WAITING: if item.task.stat is STAT_WAITING:
# CalibreTask.start() should wrap all exceptions in it's own error handling # CalibreTask.start() should wrap all exceptions in its own error handling
item.task.start(self) item.task.start(self)
# remove self_cleanup tasks and hidden "System Tasks" from list
if item.task.self_cleanup or item.hidden:
self.dequeued.remove(item)
self.queue.task_done() self.queue.task_done()
def end_task(self, task_id):
ins = self.get_instance()
for __, __, __, task, __ in ins.tasks:
if str(task.id) == str(task_id) and task.is_cancellable:
task.stat = STAT_CANCELLED if task.stat == STAT_WAITING else STAT_ENDED
class CalibreTask: class CalibreTask:
__metaclass__ = abc.ABCMeta __metaclass__ = abc.ABCMeta
@ -158,10 +172,12 @@ class CalibreTask:
self.end_time = None self.end_time = None
self.message = message self.message = message
self.id = uuid.uuid4() self.id = uuid.uuid4()
self.self_cleanup = False
self._scheduled = False
@abc.abstractmethod @abc.abstractmethod
def run(self, worker_thread): def run(self, worker_thread):
"""Provides the caller some human-readable name for this class""" """The main entry-point for this task"""
raise NotImplementedError raise NotImplementedError
@abc.abstractmethod @abc.abstractmethod
@ -169,6 +185,11 @@ class CalibreTask:
"""Provides the caller some human-readable name for this class""" """Provides the caller some human-readable name for this class"""
raise NotImplementedError raise NotImplementedError
@abc.abstractmethod
def is_cancellable(self):
"""Does this task gracefully handle being cancelled (STAT_ENDED, STAT_CANCELLED)?"""
raise NotImplementedError
def start(self, *args): def start(self, *args):
self.start_time = datetime.now() self.start_time = datetime.now()
self.stat = STAT_STARTED self.stat = STAT_STARTED
@ -178,7 +199,7 @@ class CalibreTask:
self.run(*args) self.run(*args)
except Exception as ex: except Exception as ex:
self._handleError(str(ex)) self._handleError(str(ex))
log.debug_or_exception(ex) log.error_or_exception(ex)
self.end_time = datetime.now() self.end_time = datetime.now()
@ -219,15 +240,23 @@ class CalibreTask:
We have a separate dictating this because there may be certain tasks that want to override this We have a separate dictating this because there may be certain tasks that want to override this
""" """
# By default, we're good to clean a task if it's "Done" # By default, we're good to clean a task if it's "Done"
return self.stat in (STAT_FINISH_SUCCESS, STAT_FAIL) return self.stat in (STAT_FINISH_SUCCESS, STAT_FAIL, STAT_ENDED, STAT_CANCELLED)
'''@progress.setter @property
def progress(self, x): def self_cleanup(self):
if x > 1: return self._self_cleanup
x = 1
if x < 0: @self_cleanup.setter
x = 0 def self_cleanup(self, is_self_cleanup):
self._progress = x''' self._self_cleanup = is_self_cleanup
@property
def scheduled(self):
return self._scheduled
@scheduled.setter
def scheduled(self, is_scheduled):
self._scheduled = is_scheduled
def _handleError(self, error_message): def _handleError(self, error_message):
self.stat = STAT_FAIL self.stat = STAT_FAIL

347
cps/shelf.py
View File

@ -23,7 +23,7 @@
import sys import sys
from datetime import datetime from datetime import datetime
from flask import Blueprint, flash, redirect, request, url_for from flask import Blueprint, flash, redirect, request, url_for, abort
from flask_babel import gettext as _ from flask_babel import gettext as _
from flask_login import current_user, login_required from flask_login import current_user, login_required
from sqlalchemy.exc import InvalidRequestError, OperationalError from sqlalchemy.exc import InvalidRequestError, OperationalError
@ -33,30 +33,12 @@ from . import calibre_db, config, db, logger, ub
from .render_template import render_title_template from .render_template import render_title_template
from .usermanagement import login_required_if_no_ano from .usermanagement import login_required_if_no_ano
shelf = Blueprint('shelf', __name__)
log = logger.create() log = logger.create()
shelf = Blueprint('shelf', __name__)
def check_shelf_edit_permissions(cur_shelf):
if not cur_shelf.is_public and not cur_shelf.user_id == int(current_user.id):
log.error("User %s not allowed to edit shelf %s", current_user, cur_shelf)
return False
if cur_shelf.is_public and not current_user.role_edit_shelfs():
log.info("User %s not allowed to edit public shelves", current_user)
return False
return True
def check_shelf_view_permissions(cur_shelf): @shelf.route("/shelf/add/<int:shelf_id>/<int:book_id>", methods=["POST"])
if cur_shelf.is_public:
return True
if current_user.is_anonymous or cur_shelf.user_id != current_user.id:
log.error("User is unauthorized to view non-public shelf: %s", cur_shelf)
return False
return True
@shelf.route("/shelf/add/<int:shelf_id>/<int:book_id>")
@login_required @login_required
def add_to_shelf(shelf_id, book_id): def add_to_shelf(shelf_id, book_id):
xhr = request.headers.get('X-Requested-With') == 'XMLHttpRequest' xhr = request.headers.get('X-Requested-With') == 'XMLHttpRequest'
@ -64,13 +46,13 @@ def add_to_shelf(shelf_id, book_id):
if shelf is None: if shelf is None:
log.error("Invalid shelf specified: %s", shelf_id) log.error("Invalid shelf specified: %s", shelf_id)
if not xhr: if not xhr:
flash(_(u"Invalid shelf specified"), category="error") flash(_("Invalid shelf specified"), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
return "Invalid shelf specified", 400 return "Invalid shelf specified", 400
if not check_shelf_edit_permissions(shelf): if not check_shelf_edit_permissions(shelf):
if not xhr: if not xhr:
flash(_(u"Sorry you are not allowed to add a book to that shelf"), category="error") flash(_("Sorry you are not allowed to add a book to that shelf"), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
return "Sorry you are not allowed to add a book to the that shelf", 403 return "Sorry you are not allowed to add a book to the that shelf", 403
@ -79,7 +61,7 @@ def add_to_shelf(shelf_id, book_id):
if book_in_shelf: if book_in_shelf:
log.error("Book %s is already part of %s", book_id, shelf) log.error("Book %s is already part of %s", book_id, shelf)
if not xhr: if not xhr:
flash(_(u"Book is already part of the shelf: %(shelfname)s", shelfname=shelf.name), category="error") flash(_("Book is already part of the shelf: %(shelfname)s", shelfname=shelf.name), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
return "Book is already part of the shelf: %s" % shelf.name, 400 return "Book is already part of the shelf: %s" % shelf.name, 400
@ -94,17 +76,17 @@ def add_to_shelf(shelf_id, book_id):
try: try:
ub.session.merge(shelf) ub.session.merge(shelf)
ub.session.commit() ub.session.commit()
except (OperationalError, InvalidRequestError): except (OperationalError, InvalidRequestError) as e:
ub.session.rollback() ub.session.rollback()
log.error("Settings DB is not Writeable") log.error_or_exception("Settings Database error: {}".format(e))
flash(_(u"Settings DB is not Writeable"), category="error") flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
if "HTTP_REFERER" in request.environ: if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"]) return redirect(request.environ["HTTP_REFERER"])
else: else:
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
if not xhr: if not xhr:
log.debug("Book has been added to shelf: {}".format(shelf.name)) log.debug("Book has been added to shelf: {}".format(shelf.name))
flash(_(u"Book has been added to shelf: %(sname)s", sname=shelf.name), category="success") flash(_("Book has been added to shelf: %(sname)s", sname=shelf.name), category="success")
if "HTTP_REFERER" in request.environ: if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"]) return redirect(request.environ["HTTP_REFERER"])
else: else:
@ -112,18 +94,18 @@ def add_to_shelf(shelf_id, book_id):
return "", 204 return "", 204
@shelf.route("/shelf/massadd/<int:shelf_id>") @shelf.route("/shelf/massadd/<int:shelf_id>", methods=["POST"])
@login_required @login_required
def search_to_shelf(shelf_id): def search_to_shelf(shelf_id):
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first() shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
if shelf is None: if shelf is None:
log.error("Invalid shelf specified: %s", shelf_id) log.error("Invalid shelf specified: {}".format(shelf_id))
flash(_(u"Invalid shelf specified"), category="error") flash(_("Invalid shelf specified"), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
if not check_shelf_edit_permissions(shelf): if not check_shelf_edit_permissions(shelf):
log.warning("You are not allowed to add a book to the shelf".format(shelf.name)) log.warning("You are not allowed to add a book to the shelf".format(shelf.name))
flash(_(u"You are not allowed to add a book to the shelf"), category="error") flash(_("You are not allowed to add a book to the shelf"), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
if current_user.id in ub.searched_ids and ub.searched_ids[current_user.id]: if current_user.id in ub.searched_ids and ub.searched_ids[current_user.id]:
@ -141,7 +123,7 @@ def search_to_shelf(shelf_id):
if not books_for_shelf: if not books_for_shelf:
log.error("Books are already part of {}".format(shelf.name)) log.error("Books are already part of {}".format(shelf.name))
flash(_(u"Books are already part of the shelf: %(name)s", name=shelf.name), category="error") flash(_("Books are already part of the shelf: %(name)s", name=shelf.name), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
maxOrder = ub.session.query(func.max(ub.BookShelf.order)).filter(ub.BookShelf.shelf == shelf_id).first()[0] or 0 maxOrder = ub.session.query(func.max(ub.BookShelf.order)).filter(ub.BookShelf.shelf == shelf_id).first()[0] or 0
@ -153,18 +135,18 @@ def search_to_shelf(shelf_id):
try: try:
ub.session.merge(shelf) ub.session.merge(shelf)
ub.session.commit() ub.session.commit()
flash(_(u"Books have been added to shelf: %(sname)s", sname=shelf.name), category="success") flash(_("Books have been added to shelf: %(sname)s", sname=shelf.name), category="success")
except (OperationalError, InvalidRequestError): except (OperationalError, InvalidRequestError) as e:
ub.session.rollback() ub.session.rollback()
log.error("Settings DB is not Writeable") log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Settings DB is not Writeable"), category="error") flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
else: else:
log.error("Could not add books to shelf: {}".format(shelf.name)) log.error("Could not add books to shelf: {}".format(shelf.name))
flash(_(u"Could not add books to shelf: %(sname)s", sname=shelf.name), category="error") flash(_("Could not add books to shelf: %(sname)s", sname=shelf.name), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
@shelf.route("/shelf/remove/<int:shelf_id>/<int:book_id>") @shelf.route("/shelf/remove/<int:shelf_id>/<int:book_id>", methods=["POST"])
@login_required @login_required
def remove_from_shelf(shelf_id, book_id): def remove_from_shelf(shelf_id, book_id):
xhr = request.headers.get('X-Requested-With') == 'XMLHttpRequest' xhr = request.headers.get('X-Requested-With') == 'XMLHttpRequest'
@ -197,16 +179,16 @@ def remove_from_shelf(shelf_id, book_id):
ub.session.delete(book_shelf) ub.session.delete(book_shelf)
shelf.last_modified = datetime.utcnow() shelf.last_modified = datetime.utcnow()
ub.session.commit() ub.session.commit()
except (OperationalError, InvalidRequestError): except (OperationalError, InvalidRequestError) as e:
ub.session.rollback() ub.session.rollback()
log.error("Settings DB is not Writeable") log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Settings DB is not Writeable"), category="error") flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
if "HTTP_REFERER" in request.environ: if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"]) return redirect(request.environ["HTTP_REFERER"])
else: else:
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
if not xhr: if not xhr:
flash(_(u"Book has been removed from shelf: %(sname)s", sname=shelf.name), category="success") flash(_("Book has been removed from shelf: %(sname)s", sname=shelf.name), category="success")
if "HTTP_REFERER" in request.environ: if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"]) return redirect(request.environ["HTTP_REFERER"])
else: else:
@ -215,7 +197,7 @@ def remove_from_shelf(shelf_id, book_id):
else: else:
if not xhr: if not xhr:
log.warning("You are not allowed to remove a book from shelf: {}".format(shelf.name)) log.warning("You are not allowed to remove a book from shelf: {}".format(shelf.name))
flash(_(u"Sorry you are not allowed to remove a book from this shelf"), flash(_("Sorry you are not allowed to remove a book from this shelf"),
category="error") category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
return "Sorry you are not allowed to remove a book from this shelf", 403 return "Sorry you are not allowed to remove a book from this shelf", 403
@ -224,13 +206,8 @@ def remove_from_shelf(shelf_id, book_id):
@shelf.route("/shelf/create", methods=["GET", "POST"]) @shelf.route("/shelf/create", methods=["GET", "POST"])
@login_required @login_required
def create_shelf(): def create_shelf():
if not current_user.role_edit_shelfs() and request.method == 'POST': shelf = ub.Shelf()
flash(_(u"Sorry you are not allowed to create a public shelf"), category="error") return create_edit_shelf(shelf, page_title=_("Create a Shelf"), page="shelfcreate")
return redirect(url_for('web.index'))
else:
shelf = ub.Shelf()
return create_edit_shelf(shelf, page_title=_(u"Create a Shelf"), page="shelfcreate")
@shelf.route("/shelf/edit/<int:shelf_id>", methods=["GET", "POST"]) @shelf.route("/shelf/edit/<int:shelf_id>", methods=["GET", "POST"])
@ -238,102 +215,24 @@ def create_shelf():
def edit_shelf(shelf_id): def edit_shelf(shelf_id):
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first() shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
if not check_shelf_edit_permissions(shelf): if not check_shelf_edit_permissions(shelf):
flash(_(u"Sorry you are not allowed to edit this shelf"), category="error") flash(_("Sorry you are not allowed to edit this shelf"), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
return create_edit_shelf(shelf, page_title=_(u"Edit a shelf"), page="shelfedit", shelf_id=shelf_id) return create_edit_shelf(shelf, page_title=_("Edit a shelf"), page="shelfedit", shelf_id=shelf_id)
# if shelf ID is set, we are editing a shelf @shelf.route("/shelf/delete/<int:shelf_id>", methods=["POST"])
def create_edit_shelf(shelf, page_title, page, shelf_id=False):
sync_only_selected_shelves = current_user.kobo_only_shelves_sync
# calibre_db.session.query(ub.Shelf).filter(ub.Shelf.user_id == current_user.id).filter(ub.Shelf.kobo_sync).count()
if request.method == "POST":
to_save = request.form.to_dict()
shelf.is_public = 1 if to_save.get("is_public") else 0
if config.config_kobo_sync:
shelf.kobo_sync = True if to_save.get("kobo_sync") else False
shelf_title = to_save.get("title", "")
if check_shelf_is_unique(shelf, shelf_title, shelf_id):
shelf.name = shelf_title
if not shelf_id:
shelf.user_id = int(current_user.id)
ub.session.add(shelf)
shelf_action = "created"
flash_text = _(u"Shelf %(title)s created", title=shelf_title)
else:
shelf_action = "changed"
flash_text = _(u"Shelf %(title)s changed", title=shelf_title)
try:
ub.session.commit()
log.info(u"Shelf {} {}".format(shelf_title, shelf_action))
flash(flash_text, category="success")
return redirect(url_for('shelf.show_shelf', shelf_id=shelf.id))
except (OperationalError, InvalidRequestError) as ex:
ub.session.rollback()
log.debug_or_exception(ex)
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
except Exception as ex:
ub.session.rollback()
log.debug_or_exception(ex)
flash(_(u"There was an error"), category="error")
return render_title_template('shelf_edit.html',
shelf=shelf,
title=page_title,
page=page,
kobo_sync_enabled=config.config_kobo_sync,
sync_only_selected_shelves=sync_only_selected_shelves)
def check_shelf_is_unique(shelf, title, shelf_id=False):
if shelf_id:
ident = ub.Shelf.id != shelf_id
else:
ident = true()
if shelf.is_public == 1:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 1)) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A public shelf with the name '{}' already exists.".format(title))
flash(_(u"A public shelf with the name '%(title)s' already exists.", title=title),
category="error")
else:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 0) &
(ub.Shelf.user_id == int(current_user.id))) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A private shelf with the name '{}' already exists.".format(title))
flash(_(u"A private shelf with the name '%(title)s' already exists.", title=title),
category="error")
return is_shelf_name_unique
def delete_shelf_helper(cur_shelf):
if not cur_shelf or not check_shelf_edit_permissions(cur_shelf):
return
shelf_id = cur_shelf.id
ub.session.delete(cur_shelf)
ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).delete()
ub.session.add(ub.ShelfArchive(uuid=cur_shelf.uuid, user_id=cur_shelf.user_id))
ub.session_commit("successfully deleted Shelf {}".format(cur_shelf.name))
@shelf.route("/shelf/delete/<int:shelf_id>")
@login_required @login_required
def delete_shelf(shelf_id): def delete_shelf(shelf_id):
cur_shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first() cur_shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
try: try:
delete_shelf_helper(cur_shelf) if not delete_shelf_helper(cur_shelf):
except InvalidRequestError: flash(_("Error deleting Shelf"), category="error")
else:
flash(_("Shelf successfully deleted"), category="success")
except InvalidRequestError as e:
ub.session.rollback() ub.session.rollback()
log.error("Settings DB is not Writeable") log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Settings DB is not Writeable"), category="error") flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
return redirect(url_for('web.index')) return redirect(url_for('web.index'))
@ -354,32 +253,144 @@ def show_shelf(shelf_id, sort_param, page):
@shelf.route("/shelf/order/<int:shelf_id>", methods=["GET", "POST"]) @shelf.route("/shelf/order/<int:shelf_id>", methods=["GET", "POST"])
@login_required @login_required
def order_shelf(shelf_id): def order_shelf(shelf_id):
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
if shelf and check_shelf_view_permissions(shelf):
if request.method == "POST":
to_save = request.form.to_dict()
books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).order_by(
ub.BookShelf.order.asc()).all()
counter = 0
for book in books_in_shelf:
setattr(book, 'order', to_save[str(book.book_id)])
counter += 1
# if order different from before -> shelf.last_modified = datetime.utcnow()
try:
ub.session.commit()
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
result = list()
if shelf:
result = calibre_db.session.query(db.Books) \
.join(ub.BookShelf, ub.BookShelf.book_id == db.Books.id, isouter=True) \
.add_columns(calibre_db.common_filters().label("visible")) \
.filter(ub.BookShelf.shelf == shelf_id).order_by(ub.BookShelf.order.asc()).all()
return render_title_template('shelf_order.html', entries=result,
title=_("Change order of Shelf: '%(name)s'", name=shelf.name),
shelf=shelf, page="shelforder")
else:
abort(404)
def check_shelf_edit_permissions(cur_shelf):
if not cur_shelf.is_public and not cur_shelf.user_id == int(current_user.id):
log.error("User {} not allowed to edit shelf: {}".format(current_user.id, cur_shelf.name))
return False
if cur_shelf.is_public and not current_user.role_edit_shelfs():
log.info("User {} not allowed to edit public shelves".format(current_user.id))
return False
return True
def check_shelf_view_permissions(cur_shelf):
if cur_shelf.is_public:
return True
if current_user.is_anonymous or cur_shelf.user_id != current_user.id:
log.error("User is unauthorized to view non-public shelf: {}".format(cur_shelf.name))
return False
return True
# if shelf ID is set, we are editing a shelf
def create_edit_shelf(shelf, page_title, page, shelf_id=False):
sync_only_selected_shelves = current_user.kobo_only_shelves_sync
# calibre_db.session.query(ub.Shelf).filter(ub.Shelf.user_id == current_user.id).filter(ub.Shelf.kobo_sync).count()
if request.method == "POST": if request.method == "POST":
to_save = request.form.to_dict() to_save = request.form.to_dict()
books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).order_by( if not current_user.role_edit_shelfs() and to_save.get("is_public") == "on":
ub.BookShelf.order.asc()).all() flash(_("Sorry you are not allowed to create a public shelf"), category="error")
counter = 0 return redirect(url_for('web.index'))
for book in books_in_shelf: is_public = 1 if to_save.get("is_public") == "on" else 0
setattr(book, 'order', to_save[str(book.book_id)]) if config.config_kobo_sync:
counter += 1 shelf.kobo_sync = True if to_save.get("kobo_sync") else False
# if order diffrent from before -> shelf.last_modified = datetime.utcnow() if shelf.kobo_sync:
try: ub.session.query(ub.ShelfArchive).filter(ub.ShelfArchive.user_id == current_user.id).filter(
ub.session.commit() ub.ShelfArchive.uuid == shelf.uuid).delete()
except (OperationalError, InvalidRequestError): ub.session_commit()
ub.session.rollback() shelf_title = to_save.get("title", "")
log.error("Settings DB is not Writeable") if check_shelf_is_unique(shelf_title, is_public, shelf_id):
flash(_("Settings DB is not Writeable"), category="error") shelf.name = shelf_title
shelf.is_public = is_public
if not shelf_id:
shelf.user_id = int(current_user.id)
ub.session.add(shelf)
shelf_action = "created"
flash_text = _("Shelf %(title)s created", title=shelf_title)
else:
shelf_action = "changed"
flash_text = _("Shelf %(title)s changed", title=shelf_title)
try:
ub.session.commit()
log.info("Shelf {} {}".format(shelf_title, shelf_action))
flash(flash_text, category="success")
return redirect(url_for('shelf.show_shelf', shelf_id=shelf.id))
except (OperationalError, InvalidRequestError) as ex:
ub.session.rollback()
log.error_or_exception(ex)
log.error_or_exception("Settings Database error: {}".format(ex))
flash(_("Oops! Database Error: %(error)s.", error=ex.orig), category="error")
except Exception as ex:
ub.session.rollback()
log.error_or_exception(ex)
flash(_("There was an error"), category="error")
return render_title_template('shelf_edit.html',
shelf=shelf,
title=page_title,
page=page,
kobo_sync_enabled=config.config_kobo_sync,
sync_only_selected_shelves=sync_only_selected_shelves)
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
result = list() def check_shelf_is_unique(title, is_public, shelf_id=False):
if shelf and check_shelf_view_permissions(shelf): if shelf_id:
result = calibre_db.session.query(db.Books) \ ident = ub.Shelf.id != shelf_id
.join(ub.BookShelf, ub.BookShelf.book_id == db.Books.id, isouter=True) \ else:
.add_columns(calibre_db.common_filters().label("visible")) \ ident = true()
.filter(ub.BookShelf.shelf == shelf_id).order_by(ub.BookShelf.order.asc()).all() if is_public == 1:
return render_title_template('shelf_order.html', entries=result, is_shelf_name_unique = ub.session.query(ub.Shelf) \
title=_(u"Change order of Shelf: '%(name)s'", name=shelf.name), .filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 1)) \
shelf=shelf, page="shelforder") .filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A public shelf with the name '{}' already exists.".format(title))
flash(_("A public shelf with the name '%(title)s' already exists.", title=title),
category="error")
else:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 0) &
(ub.Shelf.user_id == int(current_user.id))) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A private shelf with the name '{}' already exists.".format(title))
flash(_("A private shelf with the name '%(title)s' already exists.", title=title),
category="error")
return is_shelf_name_unique
def delete_shelf_helper(cur_shelf):
if not cur_shelf or not check_shelf_edit_permissions(cur_shelf):
return False
shelf_id = cur_shelf.id
ub.session.delete(cur_shelf)
ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).delete()
ub.session.add(ub.ShelfArchive(uuid=cur_shelf.uuid, user_id=cur_shelf.user_id))
ub.session_commit("successfully deleted Shelf {}".format(cur_shelf.name))
return True
def change_shelf_order(shelf_id, order): def change_shelf_order(shelf_id, order):
@ -399,7 +410,6 @@ def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
# check user is allowed to access shelf # check user is allowed to access shelf
if shelf and check_shelf_view_permissions(shelf): if shelf and check_shelf_view_permissions(shelf):
if shelf_type == 1: if shelf_type == 1:
# order = [ub.BookShelf.order.asc()] # order = [ub.BookShelf.order.asc()]
if sort_param == 'pubnew': if sort_param == 'pubnew':
@ -430,6 +440,7 @@ def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
db.Books, db.Books,
ub.BookShelf.shelf == shelf_id, ub.BookShelf.shelf == shelf_id,
[ub.BookShelf.order.asc()], [ub.BookShelf.order.asc()],
True, config.config_read_column,
ub.BookShelf, ub.BookShelf.book_id == db.Books.id) ub.BookShelf, ub.BookShelf.book_id == db.Books.id)
# delete chelf entries where book is not existent anymore, can happen if book is deleted outside calibre-web # delete chelf entries where book is not existent anymore, can happen if book is deleted outside calibre-web
wrong_entries = calibre_db.session.query(ub.BookShelf) \ wrong_entries = calibre_db.session.query(ub.BookShelf) \
@ -440,17 +451,17 @@ def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
try: try:
ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == entry.book_id).delete() ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == entry.book_id).delete()
ub.session.commit() ub.session.commit()
except (OperationalError, InvalidRequestError): except (OperationalError, InvalidRequestError) as e:
ub.session.rollback() ub.session.rollback()
log.error("Settings DB is not Writeable") log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Settings DB is not Writeable"), category="error") flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
return render_title_template(page, return render_title_template(page,
entries=result, entries=result,
pagination=pagination, pagination=pagination,
title=_(u"Shelf: '%(name)s'", name=shelf.name), title=_("Shelf: '%(name)s'", name=shelf.name),
shelf=shelf, shelf=shelf,
page="shelf") page="shelf")
else: else:
flash(_(u"Error opening shelf. Shelf does not exist or is not accessible"), category="error") flash(_("Error opening shelf. Shelf does not exist or is not accessible"), category="error")
return redirect(url_for("web.index")) return redirect(url_for("web.index"))

101
cps/static/css/caliBlur.css
View File

@ -3290,9 +3290,11 @@ div.btn-group[role=group][aria-label="Download, send to Kindle, reading"] .dropd
-ms-transform-origin: center top; -ms-transform-origin: center top;
transform-origin: center top; transform-origin: center top;
border: 0; border: 0;
left: 0 !important;
overflow-y: auto; overflow-y: auto;
} }
.dropdown-menu:not(.datepicker-dropdown):not(.profileDropli) {
left: 0 !important;
}
#add-to-shelves { #add-to-shelves {
max-height: calc(100% - 120px); max-height: calc(100% - 120px);
overflow-y: auto; overflow-y: auto;
@ -4423,38 +4425,6 @@ body.advanced_search > div.container-fluid > div.row-fluid > div.col-sm-10 > div
left: 49px; left: 49px;
margin-top: 5px margin-top: 5px
} }
body:not(.blur) > .navbar > .container-fluid > .navbar-header:after, body:not(.blur) > .navbar > .container-fluid > .navbar-header:before {
color: hsla(0, 0%, 100%, .7);
cursor: pointer;
display: block;
font-family: plex-icons-new, serif;
font-size: 20px;
font-stretch: 100%;
font-style: normal;
font-variant-caps: normal;
font-variant-east-asian: normal;
font-variant-numeric: normal;
font-weight: 400;
height: 60px;
letter-spacing: normal;
line-height: 60px;
position: absolute
}
body:not(.blur) > .navbar > .container-fluid > .navbar-header:before {
content: "\EA30";
-webkit-font-variant-ligatures: normal;
font-variant-ligatures: normal;
left: 20px
}
body:not(.blur) > .navbar > .container-fluid > .navbar-header:after {
content: "\EA2F";
-webkit-font-variant-ligatures: normal;
font-variant-ligatures: normal;
left: 60px
}
} }
body.admin > div.container-fluid > div > div.col-sm-10 > div.container-fluid > div.row:first-of-type > div.col > h2:before, body.admin > div.container-fluid > div > div.col-sm-10 > div.discover > h2:first-of-type:before, body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.discover > h1:before, body.newuser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.discover > h1:before { body.admin > div.container-fluid > div > div.col-sm-10 > div.container-fluid > div.row:first-of-type > div.col > h2:before, body.admin > div.container-fluid > div > div.col-sm-10 > div.discover > h2:first-of-type:before, body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.discover > h1:before, body.newuser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.discover > h1:before {
@ -5150,7 +5120,7 @@ body.login > div.navbar.navbar-default.navbar-static-top > div > div.navbar-head
pointer-events: none pointer-events: none
} }
#DeleteDomain:hover:before, #RestartDialog:hover:before, #ShutdownDialog:hover:before, #StatusDialog:hover:before, #deleteButton, #deleteModal:hover:before, body.mailset > div.container-fluid > div > div.col-sm-10 > div.discover td > a:hover { #DeleteDomain:hover:before, #RestartDialog:hover:before, #ShutdownDialog:hover:before, #StatusDialog:hover:before, #deleteButton, #deleteModal:hover:before, #cancelTaskModal:hover:before, body.mailset > div.container-fluid > div > div.col-sm-10 > div.discover td > a:hover {
cursor: pointer cursor: pointer
} }
@ -5237,7 +5207,11 @@ body.admin > div.container-fluid > div > div.col-sm-10 > div.container-fluid > d
margin-bottom: 20px margin-bottom: 20px
} }
body.admin:not(.modal-open) .btn-default { body.admin > div.container-fluid div.scheduled_tasks_details {
margin-bottom: 20px
}
body.admin .btn-default {
margin-bottom: 10px margin-bottom: 10px
} }
@ -5468,7 +5442,7 @@ body.admin.modal-open .navbar {
z-index: 0 !important z-index: 0 !important
} }
#RestartDialog, #ShutdownDialog, #StatusDialog, #deleteModal { #RestartDialog, #ShutdownDialog, #StatusDialog, #deleteModal, #cancelTaskModal {
top: 0; top: 0;
overflow: hidden; overflow: hidden;
padding-top: 70px; padding-top: 70px;
@ -5478,7 +5452,7 @@ body.admin.modal-open .navbar {
background: rgba(0, 0, 0, .5) background: rgba(0, 0, 0, .5)
} }
#RestartDialog:before, #ShutdownDialog:before, #StatusDialog:before, #deleteModal:before { #RestartDialog:before, #ShutdownDialog:before, #StatusDialog:before, #deleteModal:before, #cancelTaskModal:before {
content: "\E208"; content: "\E208";
padding-right: 10px; padding-right: 10px;
display: block; display: block;
@ -5500,18 +5474,18 @@ body.admin.modal-open .navbar {
z-index: 99 z-index: 99
} }
#RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before { #RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before, #cancelTaskModal.in:before {
-webkit-transform: translate(0, 0); -webkit-transform: translate(0, 0);
-ms-transform: translate(0, 0); -ms-transform: translate(0, 0);
transform: translate(0, 0) transform: translate(0, 0)
} }
#RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog { #RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog, #cancelTaskModal > .modal-dialog {
width: 450px; width: 450px;
margin: auto margin: auto
} }
#RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content { #RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content, #cancelTaskModal > .modal-dialog > .modal-content {
max-height: calc(100% - 90px); max-height: calc(100% - 90px);
-webkit-box-shadow: 0 5px 15px rgba(0, 0, 0, .5); -webkit-box-shadow: 0 5px 15px rgba(0, 0, 0, .5);
box-shadow: 0 5px 15px rgba(0, 0, 0, .5); box-shadow: 0 5px 15px rgba(0, 0, 0, .5);
@ -5522,7 +5496,7 @@ body.admin.modal-open .navbar {
width: 450px width: 450px
} }
#RestartDialog > .modal-dialog > .modal-content > .modal-header, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header, #StatusDialog > .modal-dialog > .modal-content > .modal-header, #deleteModal > .modal-dialog > .modal-content > .modal-header { #RestartDialog > .modal-dialog > .modal-content > .modal-header, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header, #StatusDialog > .modal-dialog > .modal-content > .modal-header, #deleteModal > .modal-dialog > .modal-content > .modal-header, #cancelTaskModal > .modal-dialog > .modal-content > .modal-header {
padding: 15px 20px; padding: 15px 20px;
border-radius: 3px 3px 0 0; border-radius: 3px 3px 0 0;
line-height: 1.71428571; line-height: 1.71428571;
@ -5535,7 +5509,7 @@ body.admin.modal-open .navbar {
text-align: left text-align: left
} }
#RestartDialog > .modal-dialog > .modal-content > .modal-header:before, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header:before, #StatusDialog > .modal-dialog > .modal-content > .modal-header:before, #deleteModal > .modal-dialog > .modal-content > .modal-header:before { #RestartDialog > .modal-dialog > .modal-content > .modal-header:before, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header:before, #StatusDialog > .modal-dialog > .modal-content > .modal-header:before, #deleteModal > .modal-dialog > .modal-content > .modal-header:before, #cancelTaskModal > .modal-dialog > .modal-content > .modal-header:before {
padding-right: 10px; padding-right: 10px;
font-size: 18px; font-size: 18px;
color: #999; color: #999;
@ -5564,6 +5538,11 @@ body.admin.modal-open .navbar {
font-family: plex-icons-new, serif font-family: plex-icons-new, serif
} }
#cancelTaskModal > .modal-dialog > .modal-content > .modal-header:before {
content: "\EA6D";
font-family: plex-icons-new, serif
}
#RestartDialog > .modal-dialog > .modal-content > .modal-header:after { #RestartDialog > .modal-dialog > .modal-content > .modal-header:after {
content: "Restart Calibre-Web"; content: "Restart Calibre-Web";
display: inline-block; display: inline-block;
@ -5588,7 +5567,13 @@ body.admin.modal-open .navbar {
font-size: 20px font-size: 20px
} }
#StatusDialog > .modal-dialog > .modal-content > .modal-header > span, #deleteModal > .modal-dialog > .modal-content > .modal-header > span, #loader > center > img, .rating-mobile { #cancelTaskModal > .modal-dialog > .modal-content > .modal-header:after {
content: "Delete Book";
display: inline-block;
font-size: 20px
}
#StatusDialog > .modal-dialog > .modal-content > .modal-header > span, #deleteModal > .modal-dialog > .modal-content > .modal-header > span, #cancelTaskModal > .modal-dialog > .modal-content > .modal-header > span, #loader > center > img, .rating-mobile {
display: none display: none
} }
@ -5602,7 +5587,7 @@ body.admin.modal-open .navbar {
text-align: left text-align: left
} }
#ShutdownDialog > .modal-dialog > .modal-content > .modal-body, #StatusDialog > .modal-dialog > .modal-content > .modal-body, #deleteModal > .modal-dialog > .modal-content > .modal-body { #ShutdownDialog > .modal-dialog > .modal-content > .modal-body, #StatusDialog > .modal-dialog > .modal-content > .modal-body, #deleteModal > .modal-dialog > .modal-content > .modal-body, #cancelTaskModal > .modal-dialog > .modal-content > .modal-body {
padding: 20px 20px 40px; padding: 20px 20px 40px;
font-size: 16px; font-size: 16px;
line-height: 1.6em; line-height: 1.6em;
@ -5612,7 +5597,7 @@ body.admin.modal-open .navbar {
text-align: left text-align: left
} }
#RestartDialog > .modal-dialog > .modal-content > .modal-body > p, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > p, #StatusDialog > .modal-dialog > .modal-content > .modal-body > p, #deleteModal > .modal-dialog > .modal-content > .modal-body > p { #RestartDialog > .modal-dialog > .modal-content > .modal-body > p, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > p, #StatusDialog > .modal-dialog > .modal-content > .modal-body > p, #deleteModal > .modal-dialog > .modal-content > .modal-body > p, #cancelTaskModal > .modal-dialog > .modal-content > .modal-body > p {
padding: 20px 20px 0 0; padding: 20px 20px 0 0;
font-size: 16px; font-size: 16px;
line-height: 1.6em; line-height: 1.6em;
@ -5621,7 +5606,7 @@ body.admin.modal-open .navbar {
background: #282828 background: #282828
} }
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart), #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown), #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default { #RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart), #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown), #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default, #cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-default {
float: right; float: right;
z-index: 9; z-index: 9;
position: relative; position: relative;
@ -5669,6 +5654,18 @@ body.admin.modal-open .navbar {
border-radius: 3px border-radius: 3px
} }
#cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-danger {
float: right;
z-index: 9;
position: relative;
margin: 0 0 0 10px;
min-width: 80px;
padding: 10px 18px;
font-size: 16px;
line-height: 1.33;
border-radius: 3px
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart) { #RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart) {
margin: 25px 0 0 10px margin: 25px 0 0 10px
} }
@ -5681,7 +5678,11 @@ body.admin.modal-open .navbar {
margin: 0 0 0 10px margin: 0 0 0 10px
} }
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart):hover, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown):hover, #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default:hover { #cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-default {
margin: 0 0 0 10px
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart):hover, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown):hover, #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default:hover, #cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-default:hover {
background-color: hsla(0, 0%, 100%, .3) background-color: hsla(0, 0%, 100%, .3)
} }
@ -7303,11 +7304,11 @@ body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.
background-color: transparent !important background-color: transparent !important
} }
#RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog { #RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog, #cancelTaskModal > .modal-dialog {
max-width: calc(100vw - 40px) max-width: calc(100vw - 40px)
} }
#RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content { #RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content, #cancelTaskModal > .modal-dialog > .modal-content {
max-width: calc(100vw - 40px); max-width: calc(100vw - 40px);
left: 0 left: 0
} }
@ -7457,7 +7458,7 @@ body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.
padding: 30px 15px padding: 30px 15px
} }
#RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before { #RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before, #cancelTaskModal.in:before {
left: auto; left: auto;
right: 34px right: 34px
} }

4
cps/static/css/caliBlur_override.css
View File

@ -22,3 +22,7 @@ body.serieslist.grid-view div.container-fluid > div > div.col-sm-10::before {
padding: 0 0; padding: 0 0;
line-height: 15px; line-height: 15px;
} }
input.datepicker {color: transparent}
input.datepicker:focus {color: transparent}
input.datepicker:focus + input {color: #555}

19
cps/static/css/epub_themes.css Normal file
View File

@ -0,0 +1,19 @@
.lightTheme {
background: #fff;
color: #000;
}
.darkTheme {
background: #202124;
color: #fff
}
.sepiaTheme {
background: #ece1ca;
color: #000;
}
.blackTheme {
background: #000;
color: #fff
}

6
cps/static/css/libs/images/findbarButton-next-dark.svg
View File

@ -1,6 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M8 12a1 1 0 0 1-.707-.293l-5-5a1 1 0 0 1 1.414-1.414L8
9.586l4.293-4.293a1 1 0 0 1 1.414 1.414l-5 5A1 1 0 0 1 8 12z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 461 B

5
cps/static/css/libs/images/findbarButton-previous-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M13 11a1 1 0 0 1-.707-.293L8 6.414l-4.293 4.293a1 1 0 0 1-1.414-1.414l5-5a1 1 0 0 1 1.414 0l5 5A1 1 0 0 1 13 11z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 458 B

BIN
cps/static/css/libs/images/grab.cur
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 326 B

BIN
cps/static/css/libs/images/grabbing.cur
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 326 B

16
cps/static/css/libs/images/secondaryToolbarButton-documentProperties-dark.svg
View File

@ -1,16 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16
16"
fill="rgba(255,255,255,1)">
<path
d="M8 16a8 8 0 1 1 8-8 8.009 8.009 0 0 1-8 8zM8 2a6 6 0 1 0 6 6 6.006 6.006 0 0 0-6-6z">
</path>
<path
d="M8 7a1 1 0 0 0-1 1v3a1 1 0 0 0 2 0V8a1 1 0 0 0-1-1z">
</path>
<circle
cx="8" cy="5" r="1.188">
</circle>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 557 B

2
cps/static/css/libs/images/secondaryToolbarButton-firstPage-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M13 13c-.3 0-.5-.1-.7-.3L8 8.4l-4.3 4.3c-.9.9-2.3-.5-1.4-1.4l5-5c.4-.4 1-.4 1.4 0l5 5c.6.6.2 1.7-.7 1.7zm0-11H3C1.7 2 1.7 4 3 4h10c1.3 0 1.3-2 0-2z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 255 B

2
cps/static/css/libs/images/secondaryToolbarButton-handTool-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M15 3.7V13c0 1.5-1.53 3-3 3H7.13c-.72 0-1.63-.5-2.13-1l-5-5s.84-1 .87-1c.13-.1.33-.2.53-.2.1 0 .3.1.4.2L4 10.6V2.7c0-.6.4-1 1-1s1 .4 1 1v4.6h1V1c0-.6.4-1 1-1s1 .4 1 1v6.3h1V1.7c0-.6.4-1 1-1s1 .4 1 1v5.7h1V3.7c0-.6.4-1 1-1s1 .4 1 1z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 339 B

2
cps/static/css/libs/images/secondaryToolbarButton-lastPage-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M8 10c-.3 0-.5-.1-.7-.3l-5-5c-.9-.9.5-2.3 1.4-1.4L8 7.6l4.3-4.3c.9-.9 2.3.5 1.4 1.4l-5 5c-.2.2-.4.3-.7.3zm5 2H3c-1.3 0-1.3 2 0 2h10c1.3 0 1.3-2 0-2z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 256 B

2
cps/static/css/libs/images/secondaryToolbarButton-rotateCcw-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M1 1a1 1 0 011 1v2.4A7 7 0 118 15a7 7 0 01-4.9-2 1 1 0 011.4-1.5 5 5 0 10-1-5.5H6a1 1 0 010 2H1a1 1 0 01-1-1V2a1 1 0 011-1z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 231 B

5
cps/static/css/libs/images/secondaryToolbarButton-rotateCw-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M15 1a1 1 0 0 0-1 1v2.418A6.995 6.995 0 1 0 8 15a6.954 6.954 0 0 0 4.95-2.05 1 1 0 0 0-1.414-1.414A5.019 5.019 0 1 1 12.549 6H10a1 1 0 0 0 0 2h5a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 521 B

2
cps/static/css/libs/images/secondaryToolbarButton-scrollHorizontal-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M0 4h1.5c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5H0zM9.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C5 4.5 5.5 4 6.5 4zM16 4h-1.5c-1 0-1.5.5-1.5 1.5v5c0 1 .5 1.5 1.5 1.5H16z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 302 B

1
cps/static/css/libs/images/secondaryToolbarButton-scrollPage.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"><path d="M9.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C5 4.5 5.5 4 6.5 4z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 171 B

2
cps/static/css/libs/images/secondaryToolbarButton-scrollVertical-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M9.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C5 4.5 5.5 4 6.5 4zM11 0v.5c0 1-.5 1.5-1.5 1.5h-3C5.5 2 5 1.5 5 .5V0h6zM11 16v-.5c0-1-.5-1.5-1.5-1.5h-3c-1 0-1.5.5-1.5 1.5v.5h6z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 307 B

2
cps/static/css/libs/images/secondaryToolbarButton-scrollWrapped-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M5.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C1 4.5 1.5 4 2.5 4zM7 0v.5C7 1.5 6.5 2 5.5 2h-3C1.5 2 1 1.5 1 .5V0h6zM7 16v-.5c0-1-.5-1.5-1.5-1.5h-3c-1 0-1.5.5-1.5 1.5v.5h6zM13.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5c0-1 .5-1.5 1.5-1.5zM15 0v.5c0 1-.5 1.5-1.5 1.5h-3C9.5 2 9 1.5 9 .5V0h6zM15 16v-.507c0-1-.5-1.5-1.5-1.5h-3C9.5 14 9 14.5 9 15.5v.5h6z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 509 B

5
cps/static/css/libs/images/secondaryToolbarButton-selectTool-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M12.408 8.217l-8.083-6.7A.2.2 0 0 0 4 1.672V12.3a.2.2 0 0 0 .333.146l2.56-2.372 1.857 3.9A1.125 1.125 0 1 0 10.782 13L8.913 9.075l3.4-.51a.2.2 0 0 0 .095-.348z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 505 B

2
cps/static/css/libs/images/secondaryToolbarButton-spreadEven-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M1.5 3.5C.5 3.5 0 4 0 5v6.5c0 1 .5 1.5 1.5 1.5h4c1 0 1.5-.5 1.5-1.5V5c0-1-.5-1.5-1.5-1.5zm2 1.2c.8 0 1.4.2 1.8.6.5.4.7 1 .7 1.7 0 .5-.2 1-.5 1.4-.2.3-.5.7-1 1l-.6.4c-.4.3-.6.4-.75.56-.15.14-.25.24-.35.44H6v1.3H1c0-.6.1-1.1.3-1.5.3-.6.7-1 1.5-1.6.7-.4 1.1-.8 1.28-1 .32-.3.42-.6.42-1 0-.3-.1-.6-.23-.8-.17-.2-.37-.3-.77-.3s-.7.1-.9.5c-.04.2-.1.5-.1.9H1.1c0-.6.1-1.1.3-1.5.4-.7 1.1-1.1 2.1-1.1zM10.54 3.54C9.5 3.54 9 4 9 5v6.5c0 1 .5 1.5 1.54 1.5h4c.96 0 1.46-.5 1.46-1.5V5c0-1-.5-1.46-1.5-1.46zm1.9.95c.7 0 1.3.2 1.7.5.4.4.6.8.6 1.4 0 .4-.1.8-.4 1.1-.2.2-.3.3-.5.4.1 0 .3.1.6.3.4.3.5.8.5 1.4 0 .6-.2 1.2-.6 1.6-.4.5-1.1.7-1.9.7-1 0-1.8-.3-2.2-1-.14-.29-.24-.69-.24-1.29h1.4c0 .3 0 .5.1.7.2.4.5.5 1 .5.3 0 .5-.1.7-.3.2-.2.3-.5.3-.8 0-.5-.2-.8-.6-.95-.2-.05-.5-.15-1-.15v-1c.5 0 .8-.1 1-.14.3-.1.5-.4.5-.9 0-.3-.1-.5-.2-.7-.2-.2-.4-.3-.7-.3-.3 0-.6.1-.75.3-.2.2-.2.5-.2.86h-1.34c0-.4.1-.7.19-1.1 0-.12.2-.32.4-.62.2-.2.4-.3.7-.4.3-.1.6-.1 1-.1z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 1.0 KiB

2
cps/static/css/libs/images/secondaryToolbarButton-spreadNone-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M6 3c-1 0-1.5.5-1.5 1.5v7c0 1 .5 1.5 1.5 1.5h4c1 0 1.5-.5 1.5-1.5v-7c0-1-.5-1.5-1.5-1.5z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 196 B

2
cps/static/css/libs/images/secondaryToolbarButton-spreadOdd-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M10.56 3.5C9.56 3.5 9 4 9 5v6.5c0 1 .5 1.5 1.5 1.5h4c1 0 1.5-.5 1.5-1.5V5c0-1-.5-1.5-1.5-1.5zm1.93 1.2c.8 0 1.4.2 1.8.64.5.4.7 1 .7 1.7 0 .5-.2 1-.5 1.44-.2.3-.6.6-1 .93l-.6.4c-.4.3-.6.4-.7.55-.1.1-.2.2-.3.4h3.2v1.27h-5c0-.5.1-1 .3-1.43.2-.49.7-1 1.5-1.54.7-.5 1.1-.8 1.3-1.02.3-.3.4-.7.4-1.05 0-.3-.1-.6-.3-.77-.2-.2-.4-.3-.7-.3-.4 0-.7.2-.9.5-.1.2-.1.5-.2.9h-1.4c0-.6.2-1.1.3-1.5.4-.7 1.1-1.1 2-1.1zM1.54 3.5C.54 3.5 0 4 0 5v6.5c0 1 .5 1.5 1.54 1.5h4c1 0 1.5-.5 1.5-1.5V5c0-1-.5-1.5-1.5-1.5zm1.8 1.125H4.5V12H3V6.9H1.3v-1c.5 0 .8 0 .97-.03.33-.07.53-.17.73-.37.1-.2.2-.3.25-.5.05-.2.05-.3.05-.3z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 705 B

2
cps/static/css/libs/images/toolbarButton-bookmark-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M4 16V2s0-1 1-1h6s1 0 1 1v14l-4-5z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 142 B

1
cps/static/css/libs/images/toolbarButton-currentOutlineItem.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"><path d="m14 9h-6c-1.3 0-1.3 2 0 2h6c1.3 0 1.3-2 0-2zm-5.2-8h-3.8c-1.3 0-1.3 2 0 2h1.7zm-6.8 0c-1 0-1.3 1-0.7 1.7 0.7 0.6 1.7 0.3 1.7-0.7 0-0.5-0.4-1-1-1zm3 8c-1 0-1.3 1-0.7 1.7 0.6 0.6 1.7 0.2 1.7-0.7 0-0.5-0.4-1-1-1zm0.3-4h-0.3c-1.4 0-1.4 2 0 2h2.3zm-3.3 0c-0.9 0-1.4 1-0.7 1.7 0.7 0.6 1.7 0.2 1.7-0.7 0-0.6-0.5-1-1-1zm12 8h-9c-1.3 0-1.3 2 0 2h9c1.3 0 1.3-2 0-2zm-12 0c-1 0-1.3 1-0.7 1.7 0.7 0.6 1.7 0.2 1.7-0.712 0-0.5-0.4-1-1-1z"/><path d="m7.37 4.838 3.93-3.911v2.138h3.629v3.546h-3.629v2.138l-3.93-3.911"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 581 B

5
cps/static/css/libs/images/toolbarButton-download-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M14 3h-2v2h2v8H2V5h7V3h-.849L6.584 1.538A2 2 0 0 0 5.219 1H2a2 2 0 0 0-2 2v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V5a2 2 0 0 0-2-2zM2 3h3.219l1.072 1H2z"></path><path d="M8.146 6.146a.5.5 0 0 0 0 .707l2 2a.5.5 0 0 0 .707 0l2-2a.5.5 0 1 0-.707-.707L11 7.293V.5a.5.5 0 0 0-1 0v6.793L8.854 6.146a.5.5 0 0 0-.708 0z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 651 B

24
cps/static/css/libs/images/toolbarButton-editorFreeText.svg Normal file
View File

@ -0,0 +1,24 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- copied from https://www.svgrepo.com/svg/255881/text -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 16 16" style="enable-background:new 0 0 16 16;" xml:space="preserve">
<g>
<g transform="scale(0.03125)">
<path d="M405.787,43.574H8.17c-4.513,0-8.17,3.658-8.17,8.17v119.83c0,4.512,3.657,8.17,8.17,8.17h32.681
c4.513,0,8.17-3.658,8.17-8.17v-24.511h95.319v119.83c0,4.512,3.657,8.17,8.17,8.17c4.513,0,8.17-3.658,8.17-8.17v-128
c0-4.512-3.657-8.17-8.17-8.17H40.851c-4.513,0-8.17,3.658-8.17,8.17v24.511H16.34V59.915h381.277v103.489h-16.34v-24.511
c0-4.512-3.657-8.17-8.17-8.17h-111.66c-4.513,0-8.17,3.658-8.17,8.17v288.681c0,4.512,3.657,8.17,8.17,8.17h57.191v16.34H95.319
v-16.34h57.191c4.513,0,8.17-3.658,8.17-8.17v-128c0-4.512-3.657-8.17-8.17-8.17c-4.513,0-8.17,3.658-8.17,8.17v119.83H87.149
c-4.513,0-8.17,3.658-8.17,8.17v32.681c0,4.512,3.657,8.17,8.17,8.17h239.66c4.513,0,8.17-3.658,8.17-8.17v-32.681
c0-4.512-3.657-8.17-8.17-8.17h-57.192v-272.34h95.319v24.511c0,4.512,3.657,8.17,8.17,8.17h32.681c4.513,0,8.17-3.658,8.17-8.17
V51.745C413.957,47.233,410.3,43.574,405.787,43.574z"/>
</g>
</g>
<g>
<g transform="scale(0.03125)">
<path d="M503.83,452.085h-24.511V59.915h24.511c4.513,0,8.17-3.658,8.17-8.17s-3.657-8.17-8.17-8.17h-65.362
c-4.513,0-8.17,3.658-8.17,8.17s3.657,8.17,8.17,8.17h24.511v392.17h-24.511c-4.513,0-8.17,3.658-8.17,8.17s3.657,8.17,8.17,8.17
h65.362c4.513,0,8.17-3.658,8.17-8.17S508.343,452.085,503.83,452.085z"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.6 KiB

9
cps/static/css/libs/images/toolbarButton-editorInk.svg Normal file
View File

@ -0,0 +1,9 @@
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE svg PUBLIC '-//W3C//DTD SVG 1.1//EN' 'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd'>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16" xmlns:xlink="http://www.w3.org/1999/xlink" enable-background="new 0 0 16 16">
<g>
<g transform="scale(0.03125)">
<path d="m455.1,137.9l-32.4,32.4-81-81.1 32.4-32.4c6.6-6.6 18.1-6.6 24.7,0l56.3,56.4c6.8,6.8 6.8,17.9 0,24.7zm-270.7,271l-81-81.1 209.4-209.7 81,81.1-209.4,209.7zm-99.7-42l60.6,60.7-84.4,23.8 23.8-84.5zm399.3-282.6l-56.3-56.4c-11-11-50.7-31.8-82.4,0l-285.3,285.5c-2.5,2.5-4.3,5.5-5.2,8.9l-43,153.1c-2,7.1 0.1,14.7 5.2,20 5.2,5.3 15.6,6.2 20,5.2l153-43.1c3.4-0.9 6.4-2.7 8.9-5.2l285.1-285.5c22.7-22.7 22.7-59.7 0-82.5z"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 804 B

1
cps/static/css/libs/images/toolbarButton-menuArrow-dark.svg
View File

@ -1 +0,0 @@
<svg width="16" height="16" xmlns="http://www.w3.org/2000/svg" fill="rgba(255,255,255,1)"><path d="M8 11a1 1 0 01-.707-.293l-2.99-2.99c-.91-.942.471-2.324 1.414-1.414L8 8.586l2.283-2.283c.943-.91 2.324.472 1.414 1.414l-2.99 2.99A1 1 0 018 11z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 251 B

5
cps/static/css/libs/images/toolbarButton-openFile-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M14.859 3.2a1.335 1.335 0 0 1-1.217.8H13v1h1v8H2V5h8V4h-.642a1.365 1.365 0 0 1-1.325-1.11L6.584 1.538A2 2 0 0 0 5.219 1H2a2 2 0 0 0-2 2v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V5a2 2 0 0 0-1.141-1.8zM2 3h3.219l1.072 1H2zm7.854-.146L11 1.707V8.5a.5.5 0 0 0 1 0V1.707l1.146 1.146a.5.5 0 1 0 .707-.707l-2-2a.5.5 0 0 0-.707 0l-2 2a.5.5 0 0 0 .707.707z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 686 B

8
cps/static/css/libs/images/toolbarButton-pageDown-dark.svg
View File

@ -1,8 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16
16"
fill="rgba(255,255,255,1)"><path transform='rotate(90) translate(0, -16)'
d="M15.707 7.293l-6-6a1 1 0 0 0-1.414 1.414L12.586 7H1a1 1 0 0 0 0 2h11.586l-4.293
4.293a1 1 0 1 0 1.414 1.414l6-6a1 1 0 0 0 0-1.414z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 517 B

13
cps/static/css/libs/images/toolbarButton-pageUp-dark.svg
View File

@ -1,13 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16
16"
fill="rgba(255,255,255,1)">
<path
transform='rotate(90) translate(0, -16)'
d="M15 7H3.414l4.293-4.293a1 1 0 0
0-1.414-1.414l-6 6a1 1 0 0 0 0 1.414l6 6a1 1 0 0 0 1.414-1.414L3.414 9H15a1 1 0 0
0 0-2z">
</path>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 517 B

2
cps/static/css/libs/images/toolbarButton-presentationMode-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M.5 1H7s0-1 1-1 1 1 1 1h6.5s.5 0 .5.5-.5.5-.5.5H.5S0 2 0 1.5.5 1 .5 1zM1 3h14v7c0 2-1 2-2 2H3c-1 0-2 0-2-2zm5 1v7l6-3.5zM3.72 15.33l.53-2s0-.5.65-.35c.51.13.38.63.38.63l-.53 2s0 .5-.64.35c-.53-.13-.39-.63-.39-.63zM11.24 15.61l-.53-1.99s0-.5.38-.63c.51-.13.64.35.64.35l.53 2s0 .5-.38.63c-.5.13-.64-.35-.65-.35z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 417 B

5
cps/static/css/libs/images/toolbarButton-print-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M14 5h-1V1a1 1 0 0 0-1-1H4a1 1 0 0 0-1 1v4H2a2 2 0 0 0-2 2v5h3v3a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1v-3h3V7a2 2 0 0 0-2-2zM2.5 8a.5.5 0 1 1 .5-.5.5.5 0 0 1-.5.5zm9.5 7H4v-5h8zm0-10H4V1h8zm-6.5 7h4a.5.5 0 0 0 0-1h-4a.5.5 0 1 0 0 1zm0 2h5a.5.5 0 0 0 0-1h-5a.5.5 0 1 0 0 1z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 610 B

5
cps/static/css/libs/images/toolbarButton-search-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M15.707 14.293l-4.822-4.822a6.019 6.019 0 1 0-1.414 1.414l4.822 4.822a1 1 0 0 0 1.414-1.414zM6 10a4 4 0 1 1 4-4 4 4 0 0 1-4 4z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 472 B

5
cps/static/css/libs/images/toolbarButton-secondaryToolbarToggle-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M8.707 7.293l-5-5a1 1 0 0 0-1.414 1.414L6.586 8l-4.293 4.293a1 1 0 1 0 1.414 1.414l5-5a1 1 0 0 0 0-1.414zm6 0l-5-5a1 1 0 0 0-1.414 1.414L12.586 8l-4.293 4.293a1 1 0 1 0 1.414 1.414l5-5a1 1 0 0 0 0-1.414z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 549 B

5
cps/static/css/libs/images/toolbarButton-sidebarToggle-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M3 1h10a3.008 3.008 0 0 1 3 3v8a3.009 3.009 0 0 1-3 3H3a3.005 3.005 0 0 1-3-3V4a3.013 3.013 0 0 1 3-3zm11 11V4a1 1 0 0 0-1-1H8v10h5a1 1 0 0 0 1-1zM2 12a1 1 0 0 0 1 1h4V3H3a1 1 0 0 0-1 1v8z"></path><path d="M3.5 5h2a.5.5 0 0 0 0-1h-2a.5.5 0 0 0 0 1zm0 2h2a.5.5 0 0 0 0-1h-2a.5.5 0 0 0 0 1zm1 2h1a.5.5 0 0 0 0-1h-1a.5.5 0 0 0 0 1z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 674 B

2
cps/static/css/libs/images/toolbarButton-viewAttachments-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M6.2 2s.5-.5 1.06 0c.5.5 0 1 0 1l-4.6 4.61s-2.5 2.5 0 5 5 0 5 0L13.8 6.4s1.6-1.6 0-3.2-3.2 0-3.2 0L5.8 8s-.7.7 0 1.4 1.4 0 1.4 0l3.9-3.9s.6-.5 1 0c.5.5 0 1 0 1l-3.8 4s-1.8 1.8-3.5 0C3 8.7 4.8 7 4.8 7l4.7-4.9s2.7-2.6 5.3 0c2.6 2.6 0 5.3 0 5.3l-6.2 6.3s-3.5 3.5-7 0 0-7 0-7z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 380 B

Some files were not shown because too many files have changed in this diff Show More