mirrors/calibre-web
1
0
Fork 0
mirror of https://github.com/janeczku/calibre-web synced 2024-11-28 12:30:00 +00:00
Code Issues Releases Wiki Activity

Merge branch 'master' of github.com:GarckaMan/calibre-web into long_strip_cbrreader

Browse Source
This commit is contained in:
GarcaMan 2023-02-11 03:25:31 +00:00
commit c4326c9495
377 changed files with 144962 additions and 94732 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitattributes vendored
View File

@ -1,4 +1,5 @@
constants.py ident export-subst
/test export-ignore
/library export-ignore
cps/static/css/libs/* linguist-vendored
cps/static/js/libs/* linguist-vendored

1
.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1 @@
custom: ["https://PayPal.Me/calibreweb",]

2
.gitignore vendored
View File

@ -23,10 +23,12 @@ vendor/
# calibre-web
*.db
*.log
cps/cache
.idea/
*.bak
*.log.*
.key
settings.yaml
gdrive_credentials

4
CONTRIBUTING.md
View File

@ -26,9 +26,9 @@ The Calibre-Web documentation is hosted in the Github [Wiki](https://github.com/
Do not open up a GitHub issue if the bug is a **security vulnerability** in Calibre-Web. Instead, please write an email to "ozzie.fernandez.isaacs@googlemail.com".
Ensure the ***bug was not already reported** by searching on GitHub under [Issues](https://github.com/janeczku/calibre-web/issues). Please also check if a solution for your problem can be found in the [wiki](https://github.com/janeczku/calibre-web/wiki).
Ensure the **bug was not already reported** by searching on GitHub under [Issues](https://github.com/janeczku/calibre-web/issues). Please also check if a solution for your problem can be found in the [wiki](https://github.com/janeczku/calibre-web/wiki).
If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new?assignees=&labels=&template=bug_report.md&title=). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue.
If you're unable to find an **open issue** addressing the problem, open a [new one](https://github.com/janeczku/calibre-web/issues/new/choose). Be sure to include a **title** and **clear description**, as much relevant information as possible, the **issue form** helps you providing the right information. Deleting the form and just pasting the stack trace doesn't speed up fixing the problem. If your issue could be resolved, consider closing the issue.
### **Feature Request**

40
README.md Normal file → Executable file
View File

@ -1,6 +1,6 @@
# About
Calibre-Web is a web app providing a clean interface for browsing, reading and downloading eBooks using an existing [Calibre](https://calibre-ebook.com) database.
Calibre-Web is a web app providing a clean interface for browsing, reading and downloading eBooks using a valid [Calibre](https://calibre-ebook.com) database.
[![GitHub License](https://img.shields.io/github/license/janeczku/calibre-web?style=flat-square)](https://github.com/janeczku/calibre-web/blob/master/LICENSE)
[![GitHub commit activity](https://img.shields.io/github/commit-activity/w/janeczku/calibre-web?logo=github&style=flat-square&label=commits)]()
@ -19,15 +19,16 @@ Calibre-Web is a web app providing a clean interface for browsing, reading and d
- full graphical setup
- User management with fine-grained per-user permissions
- Admin interface
- User Interface in brazilian, czech, dutch, english, finnish, french, german, greek, hungarian, italian, japanese, khmer, polish, russian, simplified and traditional chinese, spanish, swedish, turkish, ukrainian
- User Interface in brazilian, czech, dutch, english, finnish, french, galician, german, greek, hungarian, indonesian, italian, japanese, khmer, korean, norwegian, polish, russian, simplified and traditional chinese, spanish, swedish, turkish, ukrainian, vietnamese
- OPDS feed for eBook reader apps
- Filter and search by titles, authors, tags, series and language
- Filter and search by titles, authors, tags, series, book format and language
- Create a custom book collection (shelves)
- Support for editing eBook metadata and deleting eBooks from Calibre library
- Support for downloading eBook metadata from various sources, sources can be extended via external plugins
- Support for converting eBooks through Calibre binaries
- Restrict eBook download to logged-in users
- Support for public user registration
- Send eBooks to Kindle devices with the click of a button
- Send eBooks to E-Readers with the click of a button
- Sync your Kobo devices through Calibre-Web with your Calibre library
- Support for reading eBooks directly in the browser (.txt, .epub, .pdf, .cbr, .cbt, .cbz, .djvu)
- Upload new books in many formats, including audio formats (.mp3, .m4a, .m4b)
@ -40,23 +41,21 @@ Calibre-Web is a web app providing a clean interface for browsing, reading and d
## Installation
#### Installation via pip (recommended)
1. Install calibre web via pip with the command `pip install calibreweb` (Depending on your OS and or distro the command could also be `pip3`).
2. Optional features can also be installed via pip, please refer to [this page](https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-Windows) for details
3. Calibre-Web can be started afterwards by typing `cps` or `python3 -m cps`
1. To avoid problems with already installed python dependencies, it's recommended to create a virtual environment for Calibre-Web
2. Install Calibre-Web via pip with the command `pip install calibreweb` (Depending on your OS and or distro the command could also be `pip3`).
3. Optional features can also be installed via pip, please refer to [this page](https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-and-Windows) for details
4. Calibre-Web can be started afterwards by typing `cps`
#### Manual installation
1. Install dependencies by running `pip3 install --target vendor -r requirements.txt` (python3.x). Alternativly set up a python virtual environment.
2. Execute the command: `python3 cps.py` (or `nohup python3 cps.py` - recommended if you want to exit the terminal window)
Issues with Ubuntu:
Please note that running the above install command can fail on some versions of Ubuntu, saying `"can't combine user with prefix"`. This is a [known bug](https://github.com/pypa/pip/issues/3826) and can be remedied by using the command `pip install --system --target vendor -r requirements.txt` instead.
In the Wiki there are also examples for: a [manual installation](https://github.com/janeczku/calibre-web/wiki/Manual-installation), [installation on Linux Mint](https://github.com/janeczku/calibre-web/wiki/How-To:Install-Calibre-Web-in-Linux-Mint-19-or-20), [installation on a Cloud Provider](https://github.com/janeczku/calibre-web/wiki/How-To:-Install-Calibre-Web-on-a-Cloud-Provider).
## Quick start
Point your browser to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog
Set `Location of Calibre database` to the path of the folder where your Calibre library (metadata.db) lives, push "submit" button\
Optionally a Google Drive can be used to host the calibre library [-> Using Google Drive integration](https://github.com/janeczku/calibre-web/wiki/Configuration#using-google-drive-integration)
Go to Login page
Point your browser to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog \
Login with default admin login \
If you don't have a Calibre database already, this [database](https://github.com/janeczku/calibre-web/blob/master/library/metadata.db) can be used. **IMPORTATNT** Please move the database out of the calibre-web folder structure, as it will be overwritten during update. \
Set `Location of Calibre database` to the path of the folder where your Calibre library (metadata.db) lives, push "submit" button. \
Optionally a Google Drive can be used to host the calibre library [-> Using Google Drive integration](https://github.com/janeczku/calibre-web/wiki/G-Drive-Setup#using-google-drive-integration) \
Afterwards you can configure your Calibre-Web instance ([Basic Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#basic-configuration) and [UI Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#ui-configuration) on admin page)
#### Default admin login:
*Username:* admin\
@ -67,11 +66,14 @@ Go to Login page
python 3.5+
Optionally, to enable on-the-fly conversion from one ebook format to another when using the send-to-kindle feature, or during editing of ebooks metadata:
[Download](https://imagemagick.org/script/download.php) Imagemagick to extract covers from epubs. On Windows the additional installation of [ghostscript](https://ghostscript.com/releases/gsdnld.html) might be necessary to extract covers from pdf files. On Linux Imagemagick and Ghostscript can often be installed using the system package manager.
Optionally, to enable on-the-fly conversion from one ebook format to another when using the send-to-ereader feature, or during editing of ebooks metadata:
[Download and install](https://calibre-ebook.com/download) the Calibre desktop program for your platform and enter the folder including program name (normally /opt/calibre/ebook-convert, or C:\Program Files\calibre\ebook-convert.exe) in the field "calibre's converter tool" on the setup page.
[Download](https://github.com/pgaskin/kepubify/releases/latest) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `\opt\kepubify` Windows: `C:\Program Files\kepubify`.
[Download](https://github.com/pgaskin/kepubify/releases/latest) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `/opt/kepubify` Windows: `C:\Program Files\kepubify`.
## Docker Images

44
SECURITY.md
View File

@ -11,19 +11,35 @@ To receive fixes for security vulnerabilities it is required to always upgrade t
## History
| Fixed in | Description |CVE number |
| ---------- |---------|---------|
| 3rd July 2018 | Guest access acts as a backdoor||
| V 0.6.7 |Hardcoded secret key for sessions |CVE-2020-12627 |
| V 0.6.13|Calibre-Web Metadata cross site scripting |CVE-2021-25964|
| V 0.6.13|Name of Shelves are only visible to users who can access the corresponding shelf Thanks to @ibarrionuevo||
| V 0.6.13|JavaScript could get executed in the description field. Thanks to @ranjit-git and Hagai Wechsler (WhiteSource)||
| V 0.6.13|JavaScript could get executed in a custom column of type "comment" field ||
| V 0.6.13|JavaScript could get executed after converting a book to another format with a title containing javascript code||
| V 0.6.13|JavaScript could get executed after converting a book to another format with a username containing javascript code||
| V 0.6.13|JavaScript could get executed in the description series, categories or publishers title||
| V 0.6.13|JavaScript could get executed in the shelf title||
| V 0.6.13|Login with the old session cookie after logout. Thanks to @ibarrionuevo||
| V 0.6.14|CSRF was possible. Thanks to @mik317 and Hagai Wechsler (WhiteSource) ||
| V 0.6.14|Cross-Site Scripting vulnerability on typeahead inputs. Thanks to @notdodo||
|---------------|--------------------------------------------------------------------------------------------------------------------|---------|
| 3rd July 2018 | Guest access acts as a backdoor ||
| V 0.6.7 | Hardcoded secret key for sessions |CVE-2020-12627 |
| V 0.6.13 | Calibre-Web Metadata cross site scripting |CVE-2021-25964|
| V 0.6.13 | Name of Shelves are only visible to users who can access the corresponding shelf Thanks to @ibarrionuevo ||
| V 0.6.13 | JavaScript could get executed in the description field. Thanks to @ranjit-git and Hagai Wechsler (WhiteSource) ||
| V 0.6.13 | JavaScript could get executed in a custom column of type "comment" field ||
| V 0.6.13 | JavaScript could get executed after converting a book to another format with a title containing javascript code ||
| V 0.6.13 | JavaScript could get executed after converting a book to another format with a username containing javascript code ||
| V 0.6.13 | JavaScript could get executed in the description series, categories or publishers title ||
| V 0.6.13 | JavaScript could get executed in the shelf title ||
| V 0.6.13 | Login with the old session cookie after logout. Thanks to @ibarrionuevo ||
| V 0.6.14 | CSRF was possible. Thanks to @mik317 and Hagai Wechsler (WhiteSource) |CVE-2021-25965|
| V 0.6.14 | Migrated some routes to POST-requests (CSRF protection). Thanks to @scara31 |CVE-2021-4164|
| V 0.6.15 | Fix for "javascript:" script links in identifier. Thanks to @scara31 |CVE-2021-4170|
| V 0.6.15 | Cross-Site Scripting vulnerability on uploaded cover file names. Thanks to @ibarrionuevo ||
| V 0.6.15 | Creating public shelfs is now denied if user is missing the edit public shelf right. Thanks to @ibarrionuevo ||
| V 0.6.15 | Changed error message in case of trying to delete a shelf unauthorized. Thanks to @ibarrionuevo ||
| V 0.6.16 | JavaScript could get executed on authors page. Thanks to @alicaz |CVE-2022-0352|
| V 0.6.16 | Localhost can no longer be used to upload covers. Thanks to @scara31 |CVE-2022-0339|
| V 0.6.16 | Another case where public shelfs could be created without permission is prevented. Thanks to @nhiephon |CVE-2022-0273|
| V 0.6.16 | It's prevented to get the name of a private shelfs. Thanks to @nhiephon |CVE-2022-0405|
| V 0.6.17 | The SSRF Protection can no longer be bypassed via an HTTP redirect. Thanks to @416e6e61 |CVE-2022-0767|
| V 0.6.17 | The SSRF Protection can no longer be bypassed via 0.0.0.0 and it's ipv6 equivalent. Thanks to @r0hanSH |CVE-2022-0766|
| V 0.6.18 | Possible SQL Injection is prevented in user table Thanks to Iman Sharafaldin (Forward Security) |CVE-2022-30765|
| V 0.6.18 | The SSRF protection no longer can be bypassed by IPV6/IPV4 embedding. Thanks to @416e6e61 |CVE-2022-0939|
| V 0.6.18 | The SSRF protection no longer can be bypassed to connect to other servers in the local network. Thanks to @michaellrowley |CVE-2022-0990|
## Statement regarding Log4j (CVE-2021-44228 and related)
Calibre-web is not affected by bugs related to Log4j. Calibre-Web is a python program, therefore not using Java, and not using the Java logging feature log4j.

3
babel.cfg
View File

@ -1,3 +1,4 @@
[python: **.py]
# has to be executed with jinja2 >=2.9 to have autoescape enabled automatically
[jinja2: **/templates/**.*ml]
extensions=jinja2.ext.autoescape,jinja2.ext.with_

66
cps.py
View File

@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2012-2019 OzzieIsaacs
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@ -17,66 +17,18 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import os
import sys
# Insert local directories into path
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'vendor'))
from cps import create_app
from cps import web_server
from cps.opds import opds
from cps.web import web
from cps.jinjia import jinjia
from cps.about import about
from cps.shelf import shelf
from cps.admin import admi
from cps.gdrive import gdrive
from cps.editbooks import editbook
from cps.remotelogin import remotelogin
from cps.search_metadata import meta
from cps.error_handler import init_errorhandler
try:
from cps.kobo import kobo, get_kobo_activated
from cps.kobo_auth import kobo_auth
kobo_available = get_kobo_activated()
except (ImportError, AttributeError): # Catch also error for not installed flask-WTF (missing csrf decorator)
kobo_available = False
try:
from cps.oauth_bb import oauth
oauth_available = True
except ImportError:
oauth_available = False
def main():
app = create_app()
init_errorhandler()
app.register_blueprint(web)
app.register_blueprint(opds)
app.register_blueprint(jinjia)
app.register_blueprint(about)
app.register_blueprint(shelf)
app.register_blueprint(admi)
app.register_blueprint(remotelogin)
app.register_blueprint(meta)
app.register_blueprint(gdrive)
app.register_blueprint(editbook)
if kobo_available:
app.register_blueprint(kobo)
app.register_blueprint(kobo_auth)
if oauth_available:
app.register_blueprint(oauth)
success = web_server.start()
sys.exit(0 if success else 1)
# Add local path to sys.path so we can import cps
path = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, path)
from cps.main import main
if __name__ == '__main__':
main()

28
cps/MyLoginManager.py
View File

@ -21,14 +21,32 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from flask_login import LoginManager
from flask import session
from flask_login import LoginManager, confirm_login
from flask import session, current_app
from flask_login.utils import decode_cookie
from flask_login.signals import user_loaded_from_cookie
class MyLoginManager(LoginManager):
def _session_protection_failed(self):
sess = session._get_current_object()
_session = session._get_current_object()
ident = self._session_identifier_generator()
if(sess and not (len(sess) == 1 and sess.get('csrf_token', None))) and ident != sess.get('_id', None):
if(_session and not (len(_session) == 1
and _session.get('csrf_token', None))) and ident != _session.get('_id', None):
return super(). _session_protection_failed()
return False
def _load_user_from_remember_cookie(self, cookie):
user_id = decode_cookie(cookie)
if user_id is not None:
session["_user_id"] = user_id
session["_fresh"] = False
user = None
if self._user_callback:
user = self._user_callback(user_id)
if user is not None:
app = current_app._get_current_object()
user_loaded_from_cookie.send(app, user=user)
# if session was restored from remember me cookie make login valid
confirm_login()
return user
return None

137
cps/__init__.py
View File

@ -19,29 +19,27 @@
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
__package__ = "cps"
import sys
import os
import mimetypes
from babel import Locale as LC
from babel import negotiate_locale
from babel.core import UnknownLocaleError
from flask import Flask, request, g
from flask import Flask
from .MyLoginManager import MyLoginManager
from flask_babel import Babel
from flask_principal import Principal
from . import config_sql, logger, cache_buster, cli, ub, db
from . import logger
from .cli import CliParameter
from .constants import CONFIG_DIR
from .reverseproxy import ReverseProxied
from .server import WebServer
from .dep_check import dependency_check
try:
import lxml
lxml_present = True
except ImportError:
lxml_present = False
from .updater import Updater
from .babel import babel, get_locale
from . import config_sql
from . import cache_buster
from . import ub, db
try:
from flask_wtf.csrf import CSRFProtect
@ -49,6 +47,7 @@ try:
except ImportError:
wtf_present = False
mimetypes.init()
mimetypes.add_type('application/xhtml+xml', '.xhtml')
mimetypes.add_type('application/epub+zip', '.epub')
@ -70,6 +69,8 @@ mimetypes.add_type('application/ogg', '.oga')
mimetypes.add_type('text/css', '.css')
mimetypes.add_type('text/javascript; charset=UTF-8', '.js')
log = logger.create()
app = Flask(__name__)
app.config.update(
SESSION_COOKIE_HTTPONLY=True,
@ -78,58 +79,69 @@ app.config.update(
WTF_CSRF_SSL_STRICT=False
)
lm = MyLoginManager()
lm.login_view = 'web.login'
lm.anonymous_user = ub.Anonymous
lm.session_protection = 'strong'
config = config_sql._ConfigSQL()
cli_param = CliParameter()
if wtf_present:
csrf = CSRFProtect()
csrf.init_app(app)
else:
csrf = None
ub.init_db(cli.settingspath)
# pylint: disable=no-member
config = config_sql.load_configuration(ub.session)
calibre_db = db.CalibreDB()
web_server = WebServer()
babel = Babel()
_BABEL_TRANSLATIONS = set()
updater_thread = Updater()
log = logger.create()
from . import services
db.CalibreDB.update_config(config)
db.CalibreDB.setup_db(config.config_calibre_dir, cli.settingspath)
calibre_db = db.CalibreDB()
def create_app():
lm.login_view = 'web.login'
lm.anonymous_user = ub.Anonymous
lm.session_protection = 'strong'
if csrf:
csrf.init_app(app)
cli_param.init()
ub.init_db(cli_param.settings_path, cli_param.user_credentials)
# pylint: disable=no-member
config_sql.load_configuration(config, ub.session, cli_param)
db.CalibreDB.update_config(config)
db.CalibreDB.setup_db(config.config_calibre_dir, cli_param.settings_path)
calibre_db.init_db()
updater_thread.init_updater(config, web_server)
# Perform dry run of updater and exit afterwards
if cli_param.dry_run:
updater_thread.dry_run()
sys.exit(0)
updater_thread.start()
if sys.version_info < (3, 0):
log.info(
'*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, please update your installation to Python3 ***')
'*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, '
'please update your installation to Python3 ***')
print(
'*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, please update your installation to Python3 ***')
'*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, '
'please update your installation to Python3 ***')
web_server.stop(True)
sys.exit(5)
if not lxml_present:
log.info('*** "lxml" is needed for calibre-web to run. Please install it using pip: "pip install lxml" ***')
print('*** "lxml" is needed for calibre-web to run. Please install it using pip: "pip install lxml" ***')
web_server.stop(True)
sys.exit(6)
if not wtf_present:
log.info('*** "flask-WTF" is needed for calibre-web to run. Please install it using pip: "pip install flask-WTF" ***')
print('*** "flask-WTF" is needed for calibre-web to run. Please install it using pip: "pip install flask-WTF" ***')
log.info('*** "flask-WTF" is needed for calibre-web to run. '
'Please install it using pip: "pip install flask-WTF" ***')
print('*** "flask-WTF" is needed for calibre-web to run. '
'Please install it using pip: "pip install flask-WTF" ***')
web_server.stop(True)
sys.exit(7)
for res in dependency_check() + dependency_check(True):
log.info('*** "{}" version does not fit the requirements. Should: {}, Found: {}, please consider installing required version ***'
log.info('*** "{}" version does not meet the requirements. '
'Should: {}, Found: {}, please consider installing required version ***'
.format(res['name'],
res['target'],
res['found']))
@ -144,10 +156,13 @@ def create_app():
app.secret_key = os.getenv('SECRET_KEY', config_sql.get_flask_session_key(ub.session))
web_server.init_app(app, config)
if hasattr(babel, "localeselector"):
babel.init_app(app)
_BABEL_TRANSLATIONS.update(str(item) for item in babel.list_translations())
_BABEL_TRANSLATIONS.add('en')
babel.localeselector(get_locale)
else:
babel.init_app(app, locale_selector=get_locale)
from . import services
if services.ldap:
services.ldap.init_app(app, config)
@ -155,34 +170,12 @@ def create_app():
services.goodreads_support.connect(config.config_goodreads_api_key,
config.config_goodreads_api_secret,
config.config_use_goodreads)
config.store_calibre_uuid(calibre_db, db.Library_Id)
# Register scheduled tasks
from .schedule import register_scheduled_tasks, register_startup_tasks
register_scheduled_tasks(config.schedule_reconnect)
register_startup_tasks()
return app
@babel.localeselector
def get_locale():
# if a user is logged in, use the locale from the user settings
user = getattr(g, 'user', None)
if user is not None and hasattr(user, "locale"):
if user.name != 'Guest': # if the account is the guest account bypass the config lang settings
return user.locale
preferred = list()
if request.accept_languages:
for x in request.accept_languages.values():
try:
preferred.append(str(LC.parse(x.replace('-', '_'))))
except (UnknownLocaleError, ValueError) as e:
log.debug('Could not parse locale "%s": %s', x, e)
return negotiate_locale(preferred or ['en'], _BABEL_TRANSLATIONS)
@babel.timezoneselector
def get_timezone():
user = getattr(g, 'user', None)
return user.timezone if user else None
from .updater import Updater
updater_thread = Updater()
updater_thread.start()

128
cps/about.py
View File

@ -25,114 +25,60 @@ import platform
import sqlite3
from collections import OrderedDict
import babel, pytz, requests, sqlalchemy
import werkzeug, flask, flask_login, flask_principal, jinja2
import flask
import flask_login
import jinja2
from flask_babel import gettext as _
try:
from flask_wtf import __version__ as flaskwtf_version
except ImportError:
flaskwtf_version = _(u'not installed')
from . import db, calibre_db, converter, uploader, server, isoLanguages, constants, gdriveutils, dep_check
from . import db, calibre_db, converter, uploader, constants, dep_check
from .render_template import render_title_template
try:
from flask_login import __version__ as flask_loginVersion
except ImportError:
from flask_login.__about__ import __version__ as flask_loginVersion
try:
# pylint: disable=unused-import
import unidecode
# _() necessary to make babel aware of string for translation
unidecode_version = _(u'installed')
except ImportError:
unidecode_version = _(u'not installed')
try:
from flask_dance import __version__ as flask_danceVersion
except ImportError:
flask_danceVersion = None
try:
from greenlet import __version__ as greenlet_Version
except ImportError:
greenlet_Version = None
try:
from scholarly import scholarly
scholarly_version = _(u'installed')
except ImportError:
scholarly_version = _(u'not installed')
from . import services
about = flask.Blueprint('about', __name__)
ret = dict()
req = dep_check.load_dependencys(False)
opt = dep_check.load_dependencys(True)
modules = dict()
req = dep_check.load_dependencies(False)
opt = dep_check.load_dependencies(True)
for i in (req + opt):
ret[i[1]] = i[0]
modules[i[1]] = i[0]
modules['Jinja2'] = jinja2.__version__
modules['pySqlite'] = sqlite3.version
modules['SQLite'] = sqlite3.sqlite_version
sorted_modules = OrderedDict((sorted(modules.items(), key=lambda x: x[0].casefold())))
if not ret:
_VERSIONS = OrderedDict(
Platform = '{0[0]} {0[2]} {0[3]} {0[4]} {0[5]}'.format(platform.uname()),
Python=sys.version,
Calibre_Web=constants.STABLE_VERSION['version'] + ' - '
+ constants.NIGHTLY_VERSION[0].replace('%','%%') + ' - '
+ constants.NIGHTLY_VERSION[1].replace('%','%%'),
WebServer=server.VERSION,
Flask=flask.__version__,
Flask_Login=flask_loginVersion,
Flask_Principal=flask_principal.__version__,
Flask_WTF=flaskwtf_version,
Werkzeug=werkzeug.__version__,
Babel=babel.__version__,
Jinja2=jinja2.__version__,
Requests=requests.__version__,
SqlAlchemy=sqlalchemy.__version__,
pySqlite=sqlite3.version,
SQLite=sqlite3.sqlite_version,
iso639=isoLanguages.__version__,
pytz=pytz.__version__,
Unidecode=unidecode_version,
Scholarly=scholarly_version,
Flask_SimpleLDAP=u'installed' if bool(services.ldap) else None,
python_LDAP=services.ldapVersion if bool(services.ldapVersion) else None,
Goodreads=u'installed' if bool(services.goodreads_support) else None,
jsonschema=services.SyncToken.__version__ if bool(services.SyncToken) else None,
flask_dance=flask_danceVersion,
greenlet=greenlet_Version
)
_VERSIONS.update(gdriveutils.get_versions())
_VERSIONS.update(uploader.get_versions(True))
else:
_VERSIONS = OrderedDict(
Platform = '{0[0]} {0[2]} {0[3]} {0[4]} {0[5]}'.format(platform.uname()),
Python = sys.version,
Calibre_Web = constants.STABLE_VERSION['version'] + ' - '
+ constants.NIGHTLY_VERSION[0].replace('%', '%%') + ' - '
+ constants.NIGHTLY_VERSION[1].replace('%', '%%'),
Werkzeug = werkzeug.__version__,
Jinja2=jinja2.__version__,
pySqlite = sqlite3.version,
SQLite = sqlite3.sqlite_version,
)
_VERSIONS.update(ret)
_VERSIONS.update(uploader.get_versions(False))
def collect_stats():
_VERSIONS['ebook converter'] = _(converter.get_calibre_version())
_VERSIONS['unrar'] = _(converter.get_unrar_version())
_VERSIONS['kepubify'] = _(converter.get_kepubify_version())
if constants.NIGHTLY_VERSION[0] == "$Format:%H$":
calibre_web_version = constants.STABLE_VERSION['version']
else:
calibre_web_version = (constants.STABLE_VERSION['version'] + ' - '
+ constants.NIGHTLY_VERSION[0].replace('%', '%%') + ' - '
+ constants.NIGHTLY_VERSION[1].replace('%', '%%'))
if getattr(sys, 'frozen', False):
calibre_web_version += " - Exe-Version"
elif constants.HOME_CONFIG:
calibre_web_version += " - pyPi"
_VERSIONS = {'Calibre Web': calibre_web_version}
_VERSIONS.update(OrderedDict(
Python=sys.version,
Platform='{0[0]} {0[2]} {0[3]} {0[4]} {0[5]}'.format(platform.uname()),
))
_VERSIONS.update(uploader.get_magick_version())
_VERSIONS['Unrar'] = converter.get_unrar_version()
_VERSIONS['Ebook converter'] = converter.get_calibre_version()
_VERSIONS['Kepubify'] = converter.get_kepubify_version()
_VERSIONS.update(sorted_modules)
return _VERSIONS
@about.route("/stats")
@flask_login.login_required
def stats():
counter = calibre_db.session.query(db.Books).count()
authors = calibre_db.session.query(db.Authors).count()
categorys = calibre_db.session.query(db.Tags).count()
categories = calibre_db.session.query(db.Tags).count()
series = calibre_db.session.query(db.Series).count()
return render_title_template('stats.html', bookcounter=counter, authorcounter=authors, versions=collect_stats(),
categorycounter=categorys, seriecounter=series, title=_(u"Statistics"), page="stat")
categorycounter=categories, seriecounter=series, title=_("Statistics"), page="stat")

1266
cps/admin.py
View File

File diff suppressed because it is too large Load Diff

40
cps/babel.py Normal file
View File

@ -0,0 +1,40 @@
from babel import negotiate_locale
from flask_babel import Babel, Locale
from babel.core import UnknownLocaleError
from flask import request
from flask_login import current_user
from . import logger
log = logger.create()
babel = Babel()
def get_locale():
# if a user is logged in, use the locale from the user settings
if current_user is not None and hasattr(current_user, "locale"):
# if the account is the guest account bypass the config lang settings
if current_user.name != 'Guest':
return current_user.locale
preferred = list()
if request.accept_languages:
for x in request.accept_languages.values():
try:
preferred.append(str(Locale.parse(x.replace('-', '_'))))
except (UnknownLocaleError, ValueError) as e:
log.debug('Could not parse locale "%s": %s', x, e)
return negotiate_locale(preferred or ['en'], get_available_translations())
def get_user_locale_language(user_language):
return Locale.parse(user_language).get_language_name(get_locale())
def get_available_locale():
return [Locale('en')] + babel.list_translations()
def get_available_translations():
return set(str(item) for item in get_available_locale())

13
cps/cache_buster.py
View File

@ -47,20 +47,23 @@ def init_cache_busting(app):
for filename in filenames:
# compute version component
rooted_filename = os.path.join(dirpath, filename)
try:
with open(rooted_filename, 'rb') as f:
file_hash = hashlib.md5(f.read()).hexdigest()[:7] # nosec
# save version to tables
file_path = rooted_filename.replace(static_folder, "")
file_path = file_path.replace("\\", "/") # Convert Windows path to web path
hash_table[file_path] = file_hash
except PermissionError:
log.error("No permission to access {} file.".format(rooted_filename))
log.debug('Finished computing cache-busting values')
def bust_filename(filename):
return hash_table.get(filename, "")
def bust_filename(file_name):
return hash_table.get(file_name, "")
def unbust_filename(filename):
return filename.split("?", 1)[0]
def unbust_filename(file_name):
return file_name.split("?", 1)[0]
@app.url_defaults
# pylint: disable=unused-variable

107
cps/cli.py
View File

@ -24,82 +24,107 @@ import socket
from .constants import CONFIG_DIR as _CONFIG_DIR
from .constants import STABLE_VERSION as _STABLE_VERSION
from .constants import NIGHTLY_VERSION as _NIGHTLY_VERSION
from .constants import DEFAULT_SETTINGS_FILE, DEFAULT_GDRIVE_FILE
def version_info():
if _NIGHTLY_VERSION[1].startswith('$Format'):
return "Calibre-Web version: %s - unkown git-clone" % _STABLE_VERSION['version']
return "Calibre-Web version: %s - unknown git-clone" % _STABLE_VERSION['version']
return "Calibre-Web version: %s -%s" % (_STABLE_VERSION['version'], _NIGHTLY_VERSION[1])
parser = argparse.ArgumentParser(description='Calibre Web is a web app'
' providing a interface for browsing, reading and downloading eBooks\n', prog='cps.py')
parser.add_argument('-p', metavar='path', help='path and name to settings db, e.g. /opt/cw.db')
parser.add_argument('-g', metavar='path', help='path and name to gdrive db, e.g. /opt/gd.db')
parser.add_argument('-c', metavar='path',
help='path and name to SSL certfile, e.g. /opt/test.cert, works only in combination with keyfile')
parser.add_argument('-k', metavar='path',
help='path and name to SSL keyfile, e.g. /opt/test.key, works only in combination with certfile')
parser.add_argument('-v', '--version', action='version', help='Shows version number and exits Calibre-web',
class CliParameter(object):
def init(self):
self.arg_parser()
def arg_parser(self):
parser = argparse.ArgumentParser(description='Calibre Web is a web app providing '
'a interface for browsing, reading and downloading eBooks\n',
prog='cps.py')
parser.add_argument('-p', metavar='path', help='path and name to settings db, e.g. /opt/cw.db')
parser.add_argument('-g', metavar='path', help='path and name to gdrive db, e.g. /opt/gd.db')
parser.add_argument('-c', metavar='path', help='path and name to SSL certfile, e.g. /opt/test.cert, '
'works only in combination with keyfile')
parser.add_argument('-k', metavar='path', help='path and name to SSL keyfile, e.g. /opt/test.key, '
'works only in combination with certfile')
parser.add_argument('-v', '--version', action='version', help='Shows version number and exits Calibre-Web',
version=version_info())
parser.add_argument('-i', metavar='ip-address', help='Server IP-Address to listen')
parser.add_argument('-s', metavar='user:pass', help='Sets specific username to new password')
parser.add_argument('-f', action='store_true', help='Flag is depreciated and will be removed in next version')
args = parser.parse_args()
parser.add_argument('-i', metavar='ip-address', help='Server IP-Address to listen')
parser.add_argument('-s', metavar='user:pass',
help='Sets specific username to new password and exits Calibre-Web')
parser.add_argument('-f', action='store_true', help='Flag is depreciated and will be removed in next version')
parser.add_argument('-l', action='store_true', help='Allow loading covers from localhost')
parser.add_argument('-d', action='store_true', help='Dry run of updater to check file permissions in advance '
'and exits Calibre-Web')
parser.add_argument('-r', action='store_true', help='Enable public database reconnect route under /reconnect')
args = parser.parse_args()
settingspath = args.p or os.path.join(_CONFIG_DIR, "app.db")
gdpath = args.g or os.path.join(_CONFIG_DIR, "gdrive.db")
self.settings_path = args.p or os.path.join(_CONFIG_DIR, DEFAULT_SETTINGS_FILE)
self.gd_path = args.g or os.path.join(_CONFIG_DIR, DEFAULT_GDRIVE_FILE)
# handle and check parameter for ssl encryption
certfilepath = None
keyfilepath = None
if args.c:
if os.path.isdir(self.settings_path):
self.settings_path = os.path.join(self.settings_path, DEFAULT_SETTINGS_FILE)
if os.path.isdir(self.gd_path):
self.gd_path = os.path.join(self.gd_path, DEFAULT_GDRIVE_FILE)
# handle and check parameter for ssl encryption
self.certfilepath = None
self.keyfilepath = None
if args.c:
if os.path.isfile(args.c):
certfilepath = args.c
self.certfilepath = args.c
else:
print("Certfile path is invalid. Exiting...")
sys.exit(1)
if args.c == "":
certfilepath = ""
if args.c == "":
self.certfilepath = ""
if args.k:
if args.k:
if os.path.isfile(args.k):
keyfilepath = args.k
self.keyfilepath = args.k
else:
print("Keyfile path is invalid. Exiting...")
sys.exit(1)
if (args.k and not args.c) or (not args.k and args.c):
if (args.k and not args.c) or (not args.k and args.c):
print("Certfile and Keyfile have to be used together. Exiting...")
sys.exit(1)
if args.k == "":
keyfilepath = ""
if args.k == "":
self.keyfilepath = ""
# handle and check ip address argument
ip_address = args.i or None
if ip_address:
# dry run updater
self.dry_run = args.d or None
# enable reconnect endpoint for docker database reconnect
self.reconnect_enable = args.r or os.environ.get("CALIBRE_RECONNECT", None)
# load covers from localhost
self.allow_localhost = args.l or os.environ.get("CALIBRE_LOCALHOST", None)
# handle and check ip address argument
self.ip_address = args.i or None
if self.ip_address:
try:
# try to parse the given ip address with socket
if hasattr(socket, 'inet_pton'):
if ':' in ip_address:
socket.inet_pton(socket.AF_INET6, ip_address)
if ':' in self.ip_address:
socket.inet_pton(socket.AF_INET6, self.ip_address)
else:
socket.inet_pton(socket.AF_INET, ip_address)
socket.inet_pton(socket.AF_INET, self.ip_address)
else:
# on windows python < 3.4, inet_pton is not available
# on Windows python < 3.4, inet_pton is not available
# inet_atom only handles IPv4 addresses
socket.inet_aton(ip_address)
socket.inet_aton(self.ip_address)
except socket.error as err:
print(ip_address, ':', err)
print(self.ip_address, ':', err)
sys.exit(1)
# handle and check user password argument
user_credentials = args.s or None
if user_credentials and ":" not in user_credentials:
# handle and check user password argument
self.user_credentials = args.s or None
if self.user_credentials and ":" not in self.user_credentials:
print("No valid 'username:password' format")
sys.exit(3)
if args.f:
if args.f:
print("Warning: -f flag is depreciated and will be removed in next version")

94
cps/comic.py
View File

@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2018 OzzieIsaacs
# Copyright (C) 2018-2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@ -18,19 +18,16 @@
import os
from . import logger, isoLanguages
from . import logger, isoLanguages, cover
from .constants import BookMeta
log = logger.create()
try:
from wand.image import Image
use_IM = True
except (ImportError, RuntimeError) as e:
use_IM = False
log = logger.create()
try:
from comicapi.comicarchive import ComicArchive, MetaDataStyle
@ -51,37 +48,16 @@ except (ImportError, LookupError) as e:
use_rarfile = False
use_comic_meta = False
NO_JPEG_EXTENSIONS = ['.png', '.webp', '.bmp']
COVER_EXTENSIONS = ['.png', '.webp', '.bmp', '.jpg', '.jpeg']
def _cover_processing(tmp_file_name, img, extension):
tmp_cover_name = os.path.join(os.path.dirname(tmp_file_name), 'cover.jpg')
if use_IM:
# convert to jpg because calibre only supports jpg
if extension in NO_JPEG_EXTENSIONS:
with Image(filename=tmp_file_name) as imgc:
imgc.format = 'jpeg'
imgc.transform_colorspace('rgb')
imgc.save(tmp_cover_name)
return tmp_cover_name
if not img:
return None
with open(tmp_cover_name, 'wb') as f:
f.write(img)
return tmp_cover_name
def _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecutable):
cover_data = None
def _extract_cover_from_archive(original_file_extension, tmp_file_name, rar_executable):
cover_data = extension = None
if original_file_extension.upper() == '.CBZ':
cf = zipfile.ZipFile(tmp_file_name)
for name in cf.namelist():
ext = os.path.splitext(name)
if len(ext) > 1:
extension = ext[1].lower()
if extension in COVER_EXTENSIONS:
if extension in cover.COVER_EXTENSIONS:
cover_data = cf.read(name)
break
elif original_file_extension.upper() == '.CBT':
@ -90,44 +66,44 @@ def _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecu
ext = os.path.splitext(name)
if len(ext) > 1:
extension = ext[1].lower()
if extension in COVER_EXTENSIONS:
if extension in cover.COVER_EXTENSIONS:
cover_data = cf.extractfile(name).read()
break
elif original_file_extension.upper() == '.CBR' and use_rarfile:
try:
rarfile.UNRAR_TOOL = rarExecutable
rarfile.UNRAR_TOOL = rar_executable
cf = rarfile.RarFile(tmp_file_name)
for name in cf.getnames():
for name in cf.namelist():
ext = os.path.splitext(name)
if len(ext) > 1:
extension = ext[1].lower()
if extension in COVER_EXTENSIONS:
if extension in cover.COVER_EXTENSIONS:
cover_data = cf.read(name)
break
except Exception as ex:
log.debug('Rarfile failed with error: %s', ex)
return cover_data
log.debug('Rarfile failed with error: {}'.format(ex))
return cover_data, extension
def _extractCover(tmp_file_name, original_file_extension, rarExecutable):
def _extract_cover(tmp_file_name, original_file_extension, rar_executable):
cover_data = extension = None
if use_comic_meta:
archive = ComicArchive(tmp_file_name, rar_exe_path=rarExecutable)
archive = ComicArchive(tmp_file_name, rar_exe_path=rar_executable)
for index, name in enumerate(archive.getPageNameList()):
ext = os.path.splitext(name)
if len(ext) > 1:
extension = ext[1].lower()
if extension in COVER_EXTENSIONS:
if extension in cover.COVER_EXTENSIONS:
cover_data = archive.getPage(index)
break
else:
cover_data = _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecutable)
return _cover_processing(tmp_file_name, cover_data, extension)
cover_data, extension = _extract_cover_from_archive(original_file_extension, tmp_file_name, rar_executable)
return cover.cover_processing(tmp_file_name, cover_data, extension)
def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rarExecutable):
def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rar_executable):
if use_comic_meta:
archive = ComicArchive(tmp_file_path, rar_exe_path=rarExecutable)
archive = ComicArchive(tmp_file_path, rar_exe_path=rar_executable)
if archive.seemsToBeAComicArchive():
if archive.hasMetadata(MetaDataStyle.CIX):
style = MetaDataStyle.CIX
@ -137,34 +113,38 @@ def get_comic_info(tmp_file_path, original_file_name, original_file_extension, r
style = None
# if style is not None:
loadedMetadata = archive.readMetadata(style)
loaded_metadata = archive.readMetadata(style)
lang = loadedMetadata.language or ""
loadedMetadata.language = isoLanguages.get_lang3(lang)
lang = loaded_metadata.language or ""
loaded_metadata.language = isoLanguages.get_lang3(lang)
return BookMeta(
file_path=tmp_file_path,
extension=original_file_extension,
title=loadedMetadata.title or original_file_name,
title=loaded_metadata.title or original_file_name,
author=" & ".join([credit["person"]
for credit in loadedMetadata.credits if credit["role"] == "Writer"]) or u'Unknown',
cover=_extractCover(tmp_file_path, original_file_extension, rarExecutable),
description=loadedMetadata.comments or "",
for credit in loaded_metadata.credits if credit["role"] == "Writer"]) or 'Unknown',
cover=_extract_cover(tmp_file_path, original_file_extension, rar_executable),
description=loaded_metadata.comments or "",
tags="",
series=loadedMetadata.series or "",
series_id=loadedMetadata.issue or "",
languages=loadedMetadata.language,
publisher="")
series=loaded_metadata.series or "",
series_id=loaded_metadata.issue or "",
languages=loaded_metadata.language,
publisher="",
pubdate="",
identifiers=[])
return BookMeta(
file_path=tmp_file_path,
extension=original_file_extension,
title=original_file_name,
author=u'Unknown',
cover=_extractCover(tmp_file_path, original_file_extension, rarExecutable),
author='Unknown',
cover=_extract_cover(tmp_file_path, original_file_extension, rar_executable),
description="",
tags="",
series="",
series_id="",
languages="",
publisher="")
publisher="",
pubdate="",
identifiers=[])

94
cps/config_sql.py
View File

@ -29,12 +29,13 @@ try:
except ImportError:
from sqlalchemy.ext.declarative import declarative_base
from . import constants, cli, logger
from . import constants, logger
log = logger.create()
_Base = declarative_base()
class _Flask_Settings(_Base):
__tablename__ = 'flask_settings'
@ -62,18 +63,19 @@ class _Settings(_Base):
mail_gmail_token = Column(JSON, default={})
config_calibre_dir = Column(String)
config_calibre_uuid = Column(String)
config_port = Column(Integer, default=constants.DEFAULT_PORT)
config_external_port = Column(Integer, default=constants.DEFAULT_PORT)
config_certfile = Column(String)
config_keyfile = Column(String)
config_trustedhosts = Column(String,default='')
config_calibre_web_title = Column(String, default=u'Calibre-Web')
config_trustedhosts = Column(String, default='')
config_calibre_web_title = Column(String, default='Calibre-Web')
config_books_per_page = Column(Integer, default=60)
config_random_books = Column(Integer, default=4)
config_authors_max = Column(Integer, default=0)
config_read_column = Column(Integer, default=0)
config_title_regex = Column(String, default=r'^(A|The|An|Der|Die|Das|Den|Ein|Eine|Einen|Dem|Des|Einem|Eines)\s+')
config_mature_content_tags = Column(String, default='')
config_title_regex = Column(String, default=r'^(A|The|An|Der|Die|Das|Den|Ein|Eine|Einen|Dem|Des|Einem|Eines|Le|La|Les|L\'|Un|Une)\s+')
# config_mature_content_tags = Column(String, default='')
config_theme = Column(Integer, default=0)
config_log_level = Column(SmallInteger, default=logger.DEFAULT_LOG_LEVEL)
@ -122,7 +124,7 @@ class _Settings(_Base):
config_ldap_key_path = Column(String, default="")
config_ldap_dn = Column(String, default='dc=example,dc=org')
config_ldap_user_object = Column(String, default='uid=%s')
config_ldap_member_user_object = Column(String, default='') #
config_ldap_member_user_object = Column(String, default='')
config_ldap_openldap = Column(Boolean, default=True)
config_ldap_group_object_filter = Column(String, default='(&(objectclass=posixGroup)(cn=%s))')
config_ldap_group_members_field = Column(String, default='memberUid')
@ -133,13 +135,19 @@ class _Settings(_Base):
config_calibre = Column(String)
config_rarfile_location = Column(String, default=None)
config_upload_formats = Column(String, default=','.join(constants.EXTENSIONS_UPLOAD))
config_unicode_filename =Column(Boolean, default=False)
config_unicode_filename = Column(Boolean, default=False)
config_updatechannel = Column(Integer, default=constants.UPDATE_STABLE)
config_reverse_proxy_login_header_name = Column(String)
config_allow_reverse_proxy_header_login = Column(Boolean, default=False)
schedule_start_time = Column(Integer, default=4)
schedule_duration = Column(Integer, default=10)
schedule_generate_book_covers = Column(Boolean, default=False)
schedule_generate_series_covers = Column(Boolean, default=False)
schedule_reconnect = Column(Boolean, default=False)
def __repr__(self):
return self.__class__.__name__
@ -147,12 +155,16 @@ class _Settings(_Base):
# Class holds all application specific settings in calibre-web
class _ConfigSQL(object):
# pylint: disable=no-member
def __init__(self, session):
def __init__(self):
pass
def init_config(self, session, cli):
self._session = session
self._settings = None
self.db_configured = None
self.config_calibre_dir = None
self.load()
self.cli = cli
change = False
if self.config_converterpath == None: # pylint: disable=access-member-before-definition
@ -160,7 +172,6 @@ class _ConfigSQL(object):
self.config_converterpath = autodetect_calibre_binary()
if self.config_kepubifypath == None: # pylint: disable=access-member-before-definition
change = True
self.config_kepubifypath = autodetect_kepubify_binary()
@ -170,7 +181,6 @@ class _ConfigSQL(object):
if change:
self.save()
def _read_from_storage(self):
if self._settings is None:
log.debug("_ConfigSQL._read_from_storage")
@ -178,22 +188,21 @@ class _ConfigSQL(object):
return self._settings
def get_config_certfile(self):
if cli.certfilepath:
return cli.certfilepath
if cli.certfilepath == "":
if self.cli.certfilepath:
return self.cli.certfilepath
if self.cli.certfilepath == "":
return None
return self.config_certfile
def get_config_keyfile(self):
if cli.keyfilepath:
return cli.keyfilepath
if cli.certfilepath == "":
if self.cli.keyfilepath:
return self.cli.keyfilepath
if self.cli.certfilepath == "":
return None
return self.config_keyfile
@staticmethod
def get_config_ipaddress():
return cli.ip_address or ""
def get_config_ipaddress(self):
return self.cli.ip_address or ""
def _has_role(self, role_flag):
return constants.has_flag(self.config_default_role, role_flag)
@ -248,12 +257,14 @@ class _ConfigSQL(object):
return logger.get_level_name(self.config_log_level)
def get_mail_settings(self):
return {k:v for k, v in self.__dict__.items() if k.startswith('mail_')}
return {k: v for k, v in self.__dict__.items() if k.startswith('mail_')}
def get_mail_server_configured(self):
return bool((self.mail_server != constants.DEFAULT_MAIL_SERVER and self.mail_server_type == 0)
or (self.mail_gmail_token != {} and self.mail_server_type == 1))
def get_scheduled_task_settings(self):
return {k: v for k, v in self.__dict__.items() if k.startswith('schedule_')}
def set_from_dictionary(self, dictionary, field, convertor=None, default=None, encode=None):
"""Possibly updates a field of this object.
@ -285,13 +296,12 @@ class _ConfigSQL(object):
def toDict(self):
storage = {}
for k, v in self.__dict__.items():
if k[0] != '_' and not k.endswith("password") and not k.endswith("secret"):
if k[0] != '_' and not k.endswith("password") and not k.endswith("secret") and not k == "cli":
storage[k] = v
return storage
def load(self):
'''Load all configuration values from the underlying storage.'''
"""Load all configuration values from the underlying storage."""
s = self._read_from_storage() # type: _Settings
for k, v in s.__dict__.items():
if k[0] != '_':
@ -304,7 +314,6 @@ class _ConfigSQL(object):
have_metadata_db = bool(self.config_calibre_dir)
if have_metadata_db:
if not self.config_use_google_drive:
db_file = os.path.join(self.config_calibre_dir, 'metadata.db')
have_metadata_db = os.path.isfile(db_file)
self.db_configured = have_metadata_db
@ -325,7 +334,7 @@ class _ConfigSQL(object):
self._session.rollback()
def save(self):
'''Apply all configuration values to the underlying storage.'''
"""Apply all configuration values to the underlying storage."""
s = self._read_from_storage() # type: _Settings
for k, v in self.__dict__.items():
@ -351,6 +360,15 @@ class _ConfigSQL(object):
# self.config_calibre_dir = None
self.save()
def store_calibre_uuid(self, calibre_db, Library_table):
try:
calibre_uuid = calibre_db.session.query(Library_table).one_or_none()
if self.config_calibre_uuid != calibre_uuid.uuid:
self.config_calibre_uuid = calibre_uuid.uuid
self.save()
except AttributeError:
pass
def _migrate_table(session, orm_class):
changed = False
@ -403,6 +421,7 @@ def autodetect_calibre_binary():
return element
return ""
def autodetect_unrar_binary():
if sys.platform == "win32":
calibre_path = ["C:\\program files\\WinRar\\unRAR.exe",
@ -414,6 +433,7 @@ def autodetect_unrar_binary():
return element
return ""
def autodetect_kepubify_binary():
if sys.platform == "win32":
calibre_path = ["C:\\program files\\kepubify\\kepubify-windows-64Bit.exe",
@ -425,6 +445,7 @@ def autodetect_kepubify_binary():
return element
return ""
def _migrate_database(session):
# make sure the table is created, if it does not exist
_Base.metadata.create_all(session.bind)
@ -432,26 +453,21 @@ def _migrate_database(session):
_migrate_table(session, _Flask_Settings)
def load_configuration(session):
def load_configuration(conf, session, cli):
_migrate_database(session)
if not session.query(_Settings).count():
session.add(_Settings())
session.commit()
conf = _ConfigSQL(session)
# Migrate from global restrictions to user based restrictions
#if bool(conf.config_default_show & constants.MATURE_CONTENT) and conf.config_denied_tags == "":
# conf.config_denied_tags = conf.config_mature_content_tags
# conf.save()
# session.query(ub.User).filter(ub.User.mature_content != True). \
# update({"denied_tags": conf.config_mature_content_tags}, synchronize_session=False)
# session.commit()
return conf
# conf = _ConfigSQL()
conf.init_config(session, cli)
# return conf
def get_flask_session_key(session):
flask_settings = session.query(_Flask_Settings).one_or_none()
def get_flask_session_key(_session):
flask_settings = _session.query(_Flask_Settings).one_or_none()
if flask_settings == None:
flask_settings = _Flask_Settings(os.urandom(32))
session.add(flask_settings)
session.commit()
_session.add(flask_settings)
_session.commit()
return flask_settings.flask_session_key

47
cps/constants.py
View File

@ -21,28 +21,37 @@ import os
from collections import namedtuple
from sqlalchemy import __version__ as sql_version
sqlalchemy_version2 = ([int(x) for x in sql_version.split('.')] >= [2,0,0])
sqlalchemy_version2 = ([int(x) for x in sql_version.split('.')] >= [2, 0, 0])
# APP_MODE - production, development, or test
APP_MODE = os.environ.get('APP_MODE', 'production')
# if installed via pip this variable is set to true (empty file with name .HOMEDIR present)
HOME_CONFIG = os.path.isfile(os.path.join(os.path.dirname(os.path.abspath(__file__)), '.HOMEDIR'))
#In executables updater is not available, so variable is set to False there
# In executables updater is not available, so variable is set to False there
UPDATER_AVAILABLE = True
# Base dir is parent of current file, necessary if called from different folder
BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)),os.pardir))
BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir))
STATIC_DIR = os.path.join(BASE_DIR, 'cps', 'static')
TEMPLATES_DIR = os.path.join(BASE_DIR, 'cps', 'templates')
TRANSLATIONS_DIR = os.path.join(BASE_DIR, 'cps', 'translations')
# Cache dir - use CACHE_DIR environment variable, otherwise use the default directory: cps/cache
DEFAULT_CACHE_DIR = os.path.join(BASE_DIR, 'cps', 'cache')
CACHE_DIR = os.environ.get('CACHE_DIR', DEFAULT_CACHE_DIR)
if HOME_CONFIG:
home_dir = os.path.join(os.path.expanduser("~"),".calibre-web")
home_dir = os.path.join(os.path.expanduser("~"), ".calibre-web")
if not os.path.exists(home_dir):
os.makedirs(home_dir)
CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', home_dir)
else:
CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', BASE_DIR)
DEFAULT_SETTINGS_FILE = "app.db"
DEFAULT_GDRIVE_FILE = "gdrive.db"
ROLE_USER = 0 << 0
ROLE_ADMIN = 1 << 0
@ -134,10 +143,13 @@ del env_CALIBRE_PORT
EXTENSIONS_AUDIO = {'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'}
EXTENSIONS_CONVERT_FROM = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt','cbz','cbr']
EXTENSIONS_CONVERT_TO = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt']
EXTENSIONS_UPLOAD = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'djvu', 'prc', 'doc', 'docx',
'fb2', 'html', 'rtf', 'lit', 'odt', 'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'}
EXTENSIONS_CONVERT_FROM = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf',
'txt', 'htmlz', 'rtf', 'odt', 'cbz', 'cbr']
EXTENSIONS_CONVERT_TO = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2',
'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt']
EXTENSIONS_UPLOAD = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'djvu',
'prc', 'doc', 'docx', 'fb2', 'html', 'rtf', 'lit', 'odt', 'mp3', 'mp4', 'ogg',
'opus', 'wav', 'flac', 'm4a', 'm4b'}
def has_flag(value, bit_flag):
@ -149,16 +161,29 @@ def selected_roles(dictionary):
# :rtype: BookMeta
BookMeta = namedtuple('BookMeta', 'file_path, extension, title, author, cover, description, tags, series, '
'series_id, languages, publisher')
'series_id, languages, publisher, pubdate, identifiers')
STABLE_VERSION = {'version': '0.6.15 Beta'}
STABLE_VERSION = {'version': '0.6.20 Beta'}
NIGHTLY_VERSION = {}
NIGHTLY_VERSION = dict()
NIGHTLY_VERSION[0] = '$Format:%H$'
NIGHTLY_VERSION[1] = '$Format:%cI$'
# NIGHTLY_VERSION[0] = 'bb7d2c6273ae4560e83950d36d64533343623a57'
# NIGHTLY_VERSION[1] = '2018-09-09T10:13:08+02:00'
# CACHE
CACHE_TYPE_THUMBNAILS = 'thumbnails'
# Thumbnail Types
THUMBNAIL_TYPE_COVER = 1
THUMBNAIL_TYPE_SERIES = 2
THUMBNAIL_TYPE_AUTHOR = 3
# Thumbnails Sizes
COVER_THUMBNAIL_ORIGINAL = 0
COVER_THUMBNAIL_SMALL = 1
COVER_THUMBNAIL_MEDIUM = 2
COVER_THUMBNAIL_LARGE = 3
# clean-up the module namespace
del sys, os, namedtuple

23
cps/converter.py
View File

@ -18,7 +18,8 @@
import os
import re
from flask_babel import gettext as _
from flask_babel import lazy_gettext as N_
from . import config, logger
from .subproc_wrapper import process_wait
@ -26,10 +27,9 @@ from .subproc_wrapper import process_wait
log = logger.create()
# _() necessary to make babel aware of string for translation
_NOT_CONFIGURED = _('not configured')
_NOT_INSTALLED = _('not installed')
_EXECUTION_ERROR = _('Execution permissions missing')
# strings getting translated when used
_NOT_INSTALLED = N_('not installed')
_EXECUTION_ERROR = N_('Execution permissions missing')
def _get_command_version(path, pattern, argument=None):
@ -48,14 +48,15 @@ def _get_command_version(path, pattern, argument=None):
def get_calibre_version():
return _get_command_version(config.config_converterpath, r'ebook-convert.*\(calibre', '--version') \
or _NOT_CONFIGURED
return _get_command_version(config.config_converterpath, r'ebook-convert.*\(calibre', '--version')
def get_unrar_version():
return _get_command_version(config.config_rarfile_location, r'UNRAR.*\d') or _NOT_CONFIGURED
unrar_version = _get_command_version(config.config_rarfile_location, r'UNRAR.*\d')
if unrar_version == "not installed":
unrar_version = _get_command_version(config.config_rarfile_location, r'unrar.*\d', '-V')
return unrar_version
def get_kepubify_version():
return _get_command_version(config.config_kepubifypath, r'kepubify\s','--version') or _NOT_CONFIGURED
return _get_command_version(config.config_kepubifypath, r'kepubify\s', '--version')

48
cps/cover.py Normal file
View File

@ -0,0 +1,48 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
try:
from wand.image import Image
use_IM = True
except (ImportError, RuntimeError) as e:
use_IM = False
NO_JPEG_EXTENSIONS = ['.png', '.webp', '.bmp']
COVER_EXTENSIONS = ['.png', '.webp', '.bmp', '.jpg', '.jpeg']
def cover_processing(tmp_file_name, img, extension):
tmp_cover_name = os.path.join(os.path.dirname(tmp_file_name), 'cover.jpg')
if extension in NO_JPEG_EXTENSIONS:
if use_IM:
with Image(blob=img) as imgc:
imgc.format = 'jpeg'
imgc.transform_colorspace('rgb')
imgc.save(filename=tmp_cover_name)
return tmp_cover_name
else:
return None
if img:
with open(tmp_cover_name, 'wb') as f:
f.write(img)
return tmp_cover_name
else:
return None

490
cps/db.py
View File

@ -17,13 +17,14 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import os
import re
import ast
import json
from datetime import datetime
from urllib.parse import quote
import unidecode
from sqlite3 import OperationalError as sqliteOperationalError
from sqlalchemy import create_engine
from sqlalchemy import Table, Column, ForeignKey, CheckConstraint
from sqlalchemy import String, Integer, Boolean, TIMESTAMP, Float
@ -40,9 +41,8 @@ from sqlalchemy.pool import StaticPool
from sqlalchemy.sql.expression import and_, true, false, text, func, or_
from sqlalchemy.ext.associationproxy import association_proxy
from flask_login import current_user
from babel import Locale as LC
from babel.core import UnknownLocaleError
from flask_babel import gettext as _
from flask_babel import get_locale
from flask import flash
from . import logger, ub, isoLanguages
@ -50,11 +50,6 @@ from .pagination import Pagination
from weakref import WeakSet
try:
import unidecode
use_unidecode = True
except ImportError:
use_unidecode = False
log = logger.create()
@ -94,6 +89,12 @@ books_publishers_link = Table('books_publishers_link', Base.metadata,
)
class Library_Id(Base):
__tablename__ = 'library_id'
id = Column(Integer, primary_key=True)
uuid = Column(String, nullable=False)
class Identifiers(Base):
__tablename__ = 'identifiers'
@ -107,85 +108,91 @@ class Identifiers(Base):
self.type = id_type
self.book = book
def formatType(self):
def format_type(self):
format_type = self.type.lower()
if format_type == 'amazon':
return u"Amazon"
return "Amazon"
elif format_type.startswith("amazon_"):
return u"Amazon.{0}".format(format_type[7:])
return "Amazon.{0}".format(format_type[7:])
elif format_type == "isbn":
return u"ISBN"
return "ISBN"
elif format_type == "doi":
return u"DOI"
return "DOI"
elif format_type == "douban":
return u"Douban"
return "Douban"
elif format_type == "goodreads":
return u"Goodreads"
return "Goodreads"
elif format_type == "babelio":
return u"Babelio"
return "Babelio"
elif format_type == "google":
return u"Google Books"
return "Google Books"
elif format_type == "kobo":
return u"Kobo"
return "Kobo"
elif format_type == "litres":
return u"ЛитРес"
return "ЛитРес"
elif format_type == "issn":
return u"ISSN"
return "ISSN"
elif format_type == "isfdb":
return u"ISFDB"
return "ISFDB"
if format_type == "lubimyczytac":
return u"Lubimyczytac"
return "Lubimyczytac"
if format_type == "databazeknih":
return "Databáze knih"
else:
return self.type
def __repr__(self):
format_type = self.type.lower()
if format_type == "amazon" or format_type == "asin":
return u"https://amazon.com/dp/{0}".format(self.val)
return "https://amazon.com/dp/{0}".format(self.val)
elif format_type.startswith('amazon_'):
return u"https://amazon.{0}/dp/{1}".format(format_type[7:], self.val)
return "https://amazon.{0}/dp/{1}".format(format_type[7:], self.val)
elif format_type == "isbn":
return u"https://www.worldcat.org/isbn/{0}".format(self.val)
return "https://www.worldcat.org/isbn/{0}".format(self.val)
elif format_type == "doi":
return u"https://dx.doi.org/{0}".format(self.val)
return "https://dx.doi.org/{0}".format(self.val)
elif format_type == "goodreads":
return u"https://www.goodreads.com/book/show/{0}".format(self.val)
return "https://www.goodreads.com/book/show/{0}".format(self.val)
elif format_type == "babelio":
return u"https://www.babelio.com/livres/titre/{0}".format(self.val)
return "https://www.babelio.com/livres/titre/{0}".format(self.val)
elif format_type == "douban":
return u"https://book.douban.com/subject/{0}".format(self.val)
return "https://book.douban.com/subject/{0}".format(self.val)
elif format_type == "google":
return u"https://books.google.com/books?id={0}".format(self.val)
return "https://books.google.com/books?id={0}".format(self.val)
elif format_type == "kobo":
return u"https://www.kobo.com/ebook/{0}".format(self.val)
return "https://www.kobo.com/ebook/{0}".format(self.val)
elif format_type == "lubimyczytac":
return u"https://lubimyczytac.pl/ksiazka/{0}/ksiazka".format(self.val)
return "https://lubimyczytac.pl/ksiazka/{0}/ksiazka".format(self.val)
elif format_type == "litres":
return u"https://www.litres.ru/{0}".format(self.val)
return "https://www.litres.ru/{0}".format(self.val)
elif format_type == "issn":
return u"https://portal.issn.org/resource/ISSN/{0}".format(self.val)
return "https://portal.issn.org/resource/ISSN/{0}".format(self.val)
elif format_type == "isfdb":
return u"http://www.isfdb.org/cgi-bin/pl.cgi?{0}".format(self.val)
return "http://www.isfdb.org/cgi-bin/pl.cgi?{0}".format(self.val)
elif format_type == "databazeknih":
return "https://www.databazeknih.cz/knihy/{0}".format(self.val)
elif self.val.lower().startswith("javascript:"):
return quote(self.val)
else:
return u"{0}".format(self.val)
return "{0}".format(self.val)
class Comments(Base):
__tablename__ = 'comments'
id = Column(Integer, primary_key=True)
book = Column(Integer, ForeignKey('books.id'), nullable=False, unique=True)
text = Column(String(collation='NOCASE'), nullable=False)
book = Column(Integer, ForeignKey('books.id'), nullable=False)
def __init__(self, text, book):
self.text = text
def __init__(self, comment, book):
self.text = comment
self.book = book
def get(self):
return self.text
def __repr__(self):
return u"<Comments({0})>".format(self.text)
return "<Comments({0})>".format(self.text)
class Tags(Base):
@ -201,7 +208,7 @@ class Tags(Base):
return self.name
def __repr__(self):
return u"<Tags('{0})>".format(self.name)
return "<Tags('{0})>".format(self.name)
class Authors(Base):
@ -221,7 +228,7 @@ class Authors(Base):
return self.name
def __repr__(self):
return u"<Authors('{0},{1}{2}')>".format(self.name, self.sort, self.link)
return "<Authors('{0},{1}{2}')>".format(self.name, self.sort, self.link)
class Series(Base):
@ -239,7 +246,7 @@ class Series(Base):
return self.name
def __repr__(self):
return u"<Series('{0},{1}')>".format(self.name, self.sort)
return "<Series('{0},{1}')>".format(self.name, self.sort)
class Ratings(Base):
@ -255,7 +262,7 @@ class Ratings(Base):
return self.rating
def __repr__(self):
return u"<Ratings('{0}')>".format(self.rating)
return "<Ratings('{0}')>".format(self.rating)
class Languages(Base):
@ -274,7 +281,7 @@ class Languages(Base):
return self.lang_code
def __repr__(self):
return u"<Languages('{0}')>".format(self.lang_code)
return "<Languages('{0}')>".format(self.lang_code)
class Publishers(Base):
@ -292,7 +299,7 @@ class Publishers(Base):
return self.name
def __repr__(self):
return u"<Publishers('{0},{1}')>".format(self.name, self.sort)
return "<Publishers('{0},{1}')>".format(self.name, self.sort)
class Data(Base):
@ -316,7 +323,16 @@ class Data(Base):
return self.name
def __repr__(self):
return u"<Data('{0},{1}{2}{3}')>".format(self.book, self.format, self.uncompressed_size, self.name)
return "<Data('{0},{1}{2}{3}')>".format(self.book, self.format, self.uncompressed_size, self.name)
class Metadata_Dirtied(Base):
__tablename__ = 'metadata_dirtied'
id = Column(Integer, primary_key=True, autoincrement=True)
book = Column(Integer, ForeignKey('books.id'), nullable=False, unique=True)
def __init__(self, book):
self.book = book
class Books(Base):
@ -338,15 +354,15 @@ class Books(Base):
isbn = Column(String(collation='NOCASE'), default="")
flags = Column(Integer, nullable=False, default=1)
authors = relationship('Authors', secondary=books_authors_link, backref='books')
tags = relationship('Tags', secondary=books_tags_link, backref='books', order_by="Tags.name")
comments = relationship('Comments', backref='books')
data = relationship('Data', backref='books')
series = relationship('Series', secondary=books_series_link, backref='books')
ratings = relationship('Ratings', secondary=books_ratings_link, backref='books')
languages = relationship('Languages', secondary=books_languages_link, backref='books')
publishers = relationship('Publishers', secondary=books_publishers_link, backref='books')
identifiers = relationship('Identifiers', backref='books')
authors = relationship(Authors, secondary=books_authors_link, backref='books')
tags = relationship(Tags, secondary=books_tags_link, backref='books', order_by="Tags.name")
comments = relationship(Comments, backref='books')
data = relationship(Data, backref='books')
series = relationship(Series, secondary=books_series_link, backref='books')
ratings = relationship(Ratings, secondary=books_ratings_link, backref='books')
languages = relationship(Languages, secondary=books_languages_link, backref='books')
publishers = relationship(Publishers, secondary=books_publishers_link, backref='books')
identifiers = relationship(Identifiers, backref='books')
def __init__(self, title, sort, author_sort, timestamp, pubdate, series_index, last_modified, path, has_cover,
authors, tags, languages=None):
@ -360,18 +376,17 @@ class Books(Base):
self.path = path
self.has_cover = (has_cover != None)
def __repr__(self):
return u"<Books('{0},{1}{2}{3}{4}{5}{6}{7}{8}')>".format(self.title, self.sort, self.author_sort,
return "<Books('{0},{1}{2}{3}{4}{5}{6}{7}{8}')>".format(self.title, self.sort, self.author_sort,
self.timestamp, self.pubdate, self.series_index,
self.last_modified, self.path, self.has_cover)
@property
def atom_timestamp(self):
return (self.timestamp.strftime('%Y-%m-%dT%H:%M:%S+00:00') or '')
return self.timestamp.strftime('%Y-%m-%dT%H:%M:%S+00:00') or ''
class Custom_Columns(Base):
class CustomColumns(Base):
__tablename__ = 'custom_columns'
id = Column(Integer, primary_key=True)
@ -385,9 +400,33 @@ class Custom_Columns(Base):
normalized = Column(Boolean)
def get_display_dict(self):
display_dict = ast.literal_eval(self.display)
display_dict = json.loads(self.display)
return display_dict
def to_json(self, value, extra, sequence):
content = dict()
content['table'] = "custom_column_" + str(self.id)
content['column'] = "value"
content['datatype'] = self.datatype
content['is_multiple'] = None if not self.is_multiple else self.is_multiple
content['kind'] = "field"
content['name'] = self.name
content['search_terms'] = ['#' + self.label]
content['label'] = self.label
content['colnum'] = self.id
content['display'] = self.get_display_dict()
content['is_custom'] = True
content['is_category'] = self.datatype in ['text', 'rating', 'enumeration', 'series']
content['link_column'] = "value"
content['category_sort'] = "value"
content['is_csp'] = False
content['is_editable'] = self.editable
content['rec_index'] = sequence + 22 # toDo why ??
content['#value#'] = value
content['#extra#'] = extra
content['is_multiple2'] = {}
return json.dumps(content, ensure_ascii=False)
class AlchemyEncoder(json.JSONEncoder):
@ -429,7 +468,7 @@ class AlchemyEncoder(json.JSONEncoder):
return json.JSONEncoder.default(self, o)
class CalibreDB():
class CalibreDB:
_init = False
engine = None
config = None
@ -438,22 +477,27 @@ class CalibreDB():
# instances alive once they reach the end of their respective scopes
instances = WeakSet()
def __init__(self, expire_on_commit=True):
def __init__(self, expire_on_commit=True, init=False):
""" Initialize a new CalibreDB session
"""
self.session = None
if init:
self.init_db(expire_on_commit)
def init_db(self, expire_on_commit=True):
if self._init:
self.initSession(expire_on_commit)
self.init_session(expire_on_commit)
self.instances.add(self)
def initSession(self, expire_on_commit=True):
def init_session(self, expire_on_commit=True):
self.session = self.session_factory()
self.session.expire_on_commit = expire_on_commit
self.update_title_sort(self.config)
@classmethod
def setup_db_cc_classes(self, cc):
def setup_db_cc_classes(cls, cc):
cc_ids = []
books_custom_column_links = {}
for row in cc:
@ -524,12 +568,12 @@ class CalibreDB():
return cc_classes
@classmethod
def check_valid_db(cls, config_calibre_dir, app_db_path):
def check_valid_db(cls, config_calibre_dir, app_db_path, config_calibre_uuid):
if not config_calibre_dir:
return False
return False, False
dbpath = os.path.join(config_calibre_dir, "metadata.db")
if not os.path.exists(dbpath):
return False
return False, False
try:
check_engine = create_engine('sqlite://',
echo=False,
@ -539,10 +583,16 @@ class CalibreDB():
with check_engine.begin() as connection:
connection.execute(text("attach database '{}' as calibre;".format(dbpath)))
connection.execute(text("attach database '{}' as app_settings;".format(app_db_path)))
local_session = scoped_session(sessionmaker())
local_session.configure(bind=connection)
database_uuid = local_session().query(Library_Id).one_or_none()
# local_session.dispose()
check_engine.connect()
db_change = config_calibre_uuid != database_uuid.uuid
except Exception:
return False
return True
return False, False
return True, db_change
@classmethod
def update_config(cls, config):
@ -550,19 +600,16 @@ class CalibreDB():
@classmethod
def setup_db(cls, config_calibre_dir, app_db_path):
# cls.config = config
cls.dispose()
# toDo: if db changed -> delete shelfs, delete download books, delete read boks, kobo sync??
if not config_calibre_dir:
cls.config.invalidate()
return False
return None
dbpath = os.path.join(config_calibre_dir, "metadata.db")
if not os.path.exists(dbpath):
cls.config.invalidate()
return False
return None
try:
cls.engine = create_engine('sqlite://',
@ -578,7 +625,7 @@ class CalibreDB():
# conn.text_factory = lambda b: b.decode(errors = 'ignore') possible fix for #1302
except Exception as ex:
cls.config.invalidate(ex)
return False
return None
cls.config.db_configured = True
@ -587,16 +634,16 @@ class CalibreDB():
cc = conn.execute(text("SELECT id, datatype FROM custom_columns"))
cls.setup_db_cc_classes(cc)
except OperationalError as e:
log.debug_or_exception(e)
log.error_or_exception(e)
return None
cls.session_factory = scoped_session(sessionmaker(autocommit=False,
autoflush=True,
bind=cls.engine))
for inst in cls.instances:
inst.initSession()
inst.init_session()
cls._init = True
return True
def get_book(self, book_id):
return self.session.query(Books).filter(Books.id == book_id).first()
@ -605,30 +652,60 @@ class CalibreDB():
return self.session.query(Books).filter(Books.id == book_id). \
filter(self.common_filters(allow_show_archived)).first()
def get_book_read_archived(self, book_id, read_column, allow_show_archived=False):
if not read_column:
bd = (self.session.query(Books, ub.ReadBook.read_status, ub.ArchivedBook.is_archived).select_from(Books)
.join(ub.ReadBook, and_(ub.ReadBook.user_id == int(current_user.id), ub.ReadBook.book_id == book_id),
isouter=True))
else:
try:
read_column = cc_classes[read_column]
bd = (self.session.query(Books, read_column.value, ub.ArchivedBook.is_archived).select_from(Books)
.join(read_column, read_column.book == book_id,
isouter=True))
except (KeyError, AttributeError, IndexError):
log.error("Custom Column No.{} does not exist in calibre database".format(read_column))
# Skip linking read column and return None instead of read status
bd = self.session.query(Books, None, ub.ArchivedBook.is_archived)
return (bd.filter(Books.id == book_id)
.join(ub.ArchivedBook, and_(Books.id == ub.ArchivedBook.book_id,
int(current_user.id) == ub.ArchivedBook.user_id), isouter=True)
.filter(self.common_filters(allow_show_archived)).first())
def get_book_by_uuid(self, book_uuid):
return self.session.query(Books).filter(Books.uuid == book_uuid).first()
def get_book_format(self, book_id, file_format):
return self.session.query(Data).filter(Data.book == book_id).filter(Data.format == file_format).first()
def set_metadata_dirty(self, book_id):
if not self.session.query(Metadata_Dirtied).filter(Metadata_Dirtied.book == book_id).one_or_none():
self.session.add(Metadata_Dirtied(book_id))
def delete_dirty_metadata(self, book_id):
try:
self.session.query(Metadata_Dirtied).filter(Metadata_Dirtied.book == book_id).delete()
self.session.commit()
except (OperationalError) as e:
self.session.rollback()
log.error("Database error: {}".format(e))
# Language and content filters for displaying in the UI
def common_filters(self, allow_show_archived=False):
def common_filters(self, allow_show_archived=False, return_all_languages=False):
if not allow_show_archived:
archived_books = (
ub.session.query(ub.ArchivedBook)
archived_books = (ub.session.query(ub.ArchivedBook)
.filter(ub.ArchivedBook.user_id == int(current_user.id))
.filter(ub.ArchivedBook.is_archived == True)
.all()
)
.all())
archived_book_ids = [archived_book.book_id for archived_book in archived_books]
archived_filter = Books.id.notin_(archived_book_ids)
else:
archived_filter = true()
if current_user.filter_language() != "all":
lang_filter = Books.languages.any(Languages.lang_code == current_user.filter_language())
else:
if current_user.filter_language() == "all" or return_all_languages:
lang_filter = true()
else:
lang_filter = Books.languages.any(Languages.lang_code == current_user.filter_language())
negtags_list = current_user.list_denied_tags()
postags_list = current_user.list_allowed_tags()
neg_content_tags_filter = false() if negtags_list == [''] else Books.tags.any(Tags.name.in_(negtags_list))
@ -643,12 +720,12 @@ class CalibreDB():
neg_content_cc_filter = false() if neg_cc_list == [''] else \
getattr(Books, 'custom_column_' + str(self.config.config_restricted_column)). \
any(cc_classes[self.config.config_restricted_column].value.in_(neg_cc_list))
except (KeyError, AttributeError):
except (KeyError, AttributeError, IndexError):
pos_content_cc_filter = false()
neg_content_cc_filter = true()
log.error(u"Custom Column No.%d is not existing in calibre database",
self.config.config_restricted_column)
flash(_("Custom Column No.%(column)d is not existing in calibre database",
log.error("Custom Column No.{} does not exist in calibre database".format(
self.config.config_restricted_column))
flash(_("Custom Column No.%(column)d does not exist in calibre database",
column=self.config.config_restricted_column),
category="error")
@ -658,9 +735,31 @@ class CalibreDB():
return and_(lang_filter, pos_content_tags_filter, ~neg_content_tags_filter,
pos_content_cc_filter, ~neg_content_cc_filter, archived_filter)
def generate_linked_query(self, config_read_column, database):
if not config_read_column:
query = (self.session.query(database, ub.ArchivedBook.is_archived, ub.ReadBook.read_status)
.select_from(Books)
.outerjoin(ub.ReadBook,
and_(ub.ReadBook.user_id == int(current_user.id), ub.ReadBook.book_id == Books.id)))
else:
try:
read_column = cc_classes[config_read_column]
query = (self.session.query(database, ub.ArchivedBook.is_archived, read_column.value)
.select_from(Books)
.outerjoin(read_column, read_column.book == Books.id))
except (KeyError, AttributeError, IndexError):
log.error("Custom Column No.{} does not exist in calibre database".format(config_read_column))
# Skip linking read column and return None instead of read status
query = self.session.query(database, None, ub.ArchivedBook.is_archived)
return query.outerjoin(ub.ArchivedBook, and_(Books.id == ub.ArchivedBook.book_id,
int(current_user.id) == ub.ArchivedBook.user_id))
@staticmethod
def get_checkbox_sorted(inputlist, state, offset, limit, order):
def get_checkbox_sorted(inputlist, state, offset, limit, order, combo=False):
outcome = list()
if combo:
elementlist = {ele[0].id: ele for ele in inputlist}
else:
elementlist = {ele.id: ele for ele in inputlist}
for entry in state:
try:
@ -675,33 +774,42 @@ class CalibreDB():
return outcome[offset:offset + limit]
# Fill indexpage with all requested data from database
def fill_indexpage(self, page, pagesize, database, db_filter, order, *join):
return self.fill_indexpage_with_archived_books(page, pagesize, database, db_filter, order, False, *join)
def fill_indexpage(self, page, pagesize, database, db_filter, order,
join_archive_read=False, config_read_column=0, *join):
return self.fill_indexpage_with_archived_books(page, database, pagesize, db_filter, order, False,
join_archive_read, config_read_column, *join)
def fill_indexpage_with_archived_books(self, page, pagesize, database, db_filter, order, allow_show_archived,
*join):
def fill_indexpage_with_archived_books(self, page, database, pagesize, db_filter, order, allow_show_archived,
join_archive_read, config_read_column, *join):
pagesize = pagesize or self.config.config_books_per_page
if current_user.show_detail_random():
randm = self.session.query(Books) \
.filter(self.common_filters(allow_show_archived)) \
.order_by(func.random()) \
.limit(self.config.config_random_books).all()
random_query = self.generate_linked_query(config_read_column, database)
randm = (random_query.filter(self.common_filters(allow_show_archived))
.order_by(func.random())
.limit(self.config.config_random_books).all())
else:
randm = false()
off = int(int(pagesize) * (page - 1))
if join_archive_read:
query = self.generate_linked_query(config_read_column, database)
else:
query = self.session.query(database)
if len(join) == 6:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4]).outerjoin(join[5])
if len(join) == 5:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4])
if len(join) == 4:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3])
if len(join) == 3:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2])
elif len(join) == 2:
query = query.outerjoin(join[0], join[1])
elif len(join) == 1:
query = query.outerjoin(join[0])
off = int(int(pagesize) * (page - 1))
indx = len(join)
element = 0
while indx:
if indx >= 3:
query = query.outerjoin(join[element], join[element+1]).outerjoin(join[element+2])
indx -= 3
element += 3
elif indx == 2:
query = query.outerjoin(join[element], join[element+1])
indx -= 2
element += 2
elif indx == 1:
query = query.outerjoin(join[element])
indx -= 1
element += 1
query = query.filter(db_filter)\
.filter(self.common_filters(allow_show_archived))
entries = list()
@ -711,29 +819,48 @@ class CalibreDB():
len(query.all()))
entries = query.order_by(*order).offset(off).limit(pagesize).all()
except Exception as ex:
log.debug_or_exception(ex)
#for book in entries:
# book = self.order_authors(book)
log.error_or_exception(ex)
# display authors in right order
entries = self.order_authors(entries, True, join_archive_read)
return entries, randm, pagination
# Orders all Authors in the list according to authors sort
def order_authors(self, entry):
def order_authors(self, entries, list_return=False, combined=False):
# entries_copy = copy.deepcopy(entries)
# entries_copy =[]
for entry in entries:
if combined:
sort_authors = entry.Books.author_sort.split('&')
ids = [a.id for a in entry.Books.authors]
else:
sort_authors = entry.author_sort.split('&')
authors_ordered = list()
error = False
ids = [a.id for a in entry.authors]
authors_ordered = list()
# error = False
for auth in sort_authors:
results = self.session.query(Authors).filter(Authors.sort == auth.lstrip().strip()).all()
# ToDo: How to handle not found authorname
# ToDo: How to handle not found author name
if not len(results):
error = True
log.error("Author {} not found to display name in right order".format(auth.strip()))
# error = True
break
for r in results:
if r.id in ids:
authors_ordered.append(r)
if not error:
entry.authors = authors_ordered
return entry
ids.remove(r.id)
for author_id in ids:
result = self.session.query(Authors).filter(Authors.id == author_id).first()
authors_ordered.append(result)
if list_return:
if combined:
entry.Books.authors = authors_ordered
else:
entry.ordered_authors = authors_ordered
else:
return authors_ordered
return entries
def get_typeahead(self, database, query, replace=('', ''), tag_filter=true()):
query = query or ''
@ -747,21 +874,21 @@ class CalibreDB():
def check_exists_book(self, authr, title):
self.session.connection().connection.connection.create_function("lower", 1, lcase)
q = list()
authorterms = re.split(r'\s*&\s*', authr)
for authorterm in authorterms:
q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + authorterm + "%")))
author_terms = re.split(r'\s*&\s*', authr)
for author_term in author_terms:
q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + author_term + "%")))
return self.session.query(Books) \
.filter(and_(Books.authors.any(and_(*q)), func.lower(Books.title).ilike("%" + title + "%"))).first()
def search_query(self, term, *join):
def search_query(self, term, config, *join):
term.strip().lower()
self.session.connection().connection.connection.create_function("lower", 1, lcase)
q = list()
authorterms = re.split("[, ]+", term)
for authorterm in authorterms:
q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + authorterm + "%")))
query = self.session.query(Books)
author_terms = re.split("[, ]+", term)
for author_term in author_terms:
q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + author_term + "%")))
query = self.generate_linked_query(config.config_read_column, Books)
if len(join) == 6:
query = query.outerjoin(join[0], join[1]).outerjoin(join[2]).outerjoin(join[3], join[4]).outerjoin(join[5])
if len(join) == 3:
@ -770,19 +897,42 @@ class CalibreDB():
query = query.outerjoin(join[0], join[1])
elif len(join) == 1:
query = query.outerjoin(join[0])
return query.filter(self.common_filters(True)).filter(
or_(Books.tags.any(func.lower(Tags.name).ilike("%" + term + "%")),
cc = self.get_cc_columns(config, filter_config_custom_read=True)
filter_expression = [Books.tags.any(func.lower(Tags.name).ilike("%" + term + "%")),
Books.series.any(func.lower(Series.name).ilike("%" + term + "%")),
Books.authors.any(and_(*q)),
Books.publishers.any(func.lower(Publishers.name).ilike("%" + term + "%")),
func.lower(Books.title).ilike("%" + term + "%")
))
func.lower(Books.title).ilike("%" + term + "%")]
for c in cc:
if c.datatype not in ["datetime", "rating", "bool", "int", "float"]:
filter_expression.append(
getattr(Books,
'custom_column_' + str(c.id)).any(
func.lower(cc_classes[c.id].value).ilike("%" + term + "%")))
return query.filter(self.common_filters(True)).filter(or_(*filter_expression))
def get_cc_columns(self, config, filter_config_custom_read=False):
tmp_cc = self.session.query(CustomColumns).filter(CustomColumns.datatype.notin_(cc_exceptions)).all()
cc = []
r = None
if config.config_columns_to_ignore:
r = re.compile(config.config_columns_to_ignore)
for col in tmp_cc:
if filter_config_custom_read and config.config_read_column and config.config_read_column == col.id:
continue
if r and r.match(col.name):
continue
cc.append(col)
return cc
# read search results from calibre-database and return it (function is used for feed and simple search
def get_search_results(self, term, offset=None, order=None, limit=None, *join):
def get_search_results(self, term, config, offset=None, order=None, limit=None, *join):
order = order[0] if order else [Books.sort]
pagination = None
result = self.search_query(term, *join).order_by(*order).all()
result = self.search_query(term, config, *join).order_by(*order).all()
result_count = len(result)
if offset != None and limit != None:
offset = int(offset)
@ -792,22 +942,44 @@ class CalibreDB():
offset = 0
limit_all = result_count
ub.store_ids(result)
return result[offset:limit_all], result_count, pagination
ub.store_combo_ids(result)
entries = self.order_authors(result[offset:limit_all], list_return=True, combined=True)
return entries, result_count, pagination
# Creates for all stored languages a translated speaking name in the array for the UI
def speaking_language(self, languages=None):
from . import get_locale
def speaking_language(self, languages=None, return_all_languages=False, with_count=False, reverse_order=False):
if with_count:
if not languages:
languages = self.session.query(Languages, func.count('books_languages_link.book'))\
.join(books_languages_link).join(Books)\
.filter(self.common_filters(return_all_languages=return_all_languages)) \
.group_by(text('books_languages_link.lang_code')).all()
tags = list()
for lang in languages:
tag = Category(isoLanguages.get_language_name(get_locale(), lang[0].lang_code), lang[0].lang_code)
tags.append([tag, lang[1]])
# Append all books without language to list
if not return_all_languages:
no_lang_count = (self.session.query(Books)
.outerjoin(books_languages_link).outerjoin(Languages)
.filter(Languages.lang_code == None)
.filter(self.common_filters())
.count())
if no_lang_count:
tags.append([Category(_("None"), "none"), no_lang_count])
return sorted(tags, key=lambda x: x[0].name.lower(), reverse=reverse_order)
else:
if not languages:
languages = self.session.query(Languages) \
.join(books_languages_link) \
.join(Books) \
.filter(self.common_filters()) \
.filter(self.common_filters(return_all_languages=return_all_languages)) \
.group_by(text('books_languages_link.lang_code')).all()
for lang in languages:
lang.name = isoLanguages.get_language_name(get_locale(), lang.lang_code)
return languages
return sorted(languages, key=lambda x: x.name, reverse=reverse_order)
def update_title_sort(self, config, conn=None):
# user defined sort function for calibre databases (Series, etc.)
@ -821,7 +993,10 @@ class CalibreDB():
return title.strip()
conn = conn or self.session.connection().connection.connection
try:
conn.create_function("title_sort", 1, _title_sort)
except sqliteOperationalError:
pass
@classmethod
def dispose(cls):
@ -866,6 +1041,25 @@ def lcase(s):
try:
return unidecode.unidecode(s.lower())
except Exception as ex:
log = logger.create()
log.debug_or_exception(ex)
_log = logger.create()
_log.error_or_exception(ex)
return s.lower()
class Category:
name = None
id = None
count = None
rating = None
def __init__(self, name, cat_id, rating=None):
self.name = name
self.id = cat_id
self.rating = rating
self.count = 1
'''class Count:
count = None
def __init__(self, count):
self.count = count'''

12
cps/debug_info.py
View File

@ -22,6 +22,7 @@ import glob
import zipfile
import json
from io import BytesIO
from flask_babel.speaklater import LazyString
import os
@ -32,6 +33,13 @@ from .about import collect_stats
log = logger.create()
class lazyEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, LazyString):
return str(obj)
# Let the base class default method raise the TypeError
return json.JSONEncoder.default(self, obj)
def assemble_logfiles(file_name):
log_list = sorted(glob.glob(file_name + '*'), reverse=True)
wfd = BytesIO()
@ -57,8 +65,8 @@ def send_debug():
file_list.remove(element)
memory_zip = BytesIO()
with zipfile.ZipFile(memory_zip, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
zf.writestr('settings.txt', json.dumps(config.toDict()))
zf.writestr('libs.txt', json.dumps(collect_stats()))
zf.writestr('settings.txt', json.dumps(config.toDict(), sort_keys=True, indent=2))
zf.writestr('libs.txt', json.dumps(collect_stats(), sort_keys=True, indent=2, cls=lazyEncoder))
for fp in file_list:
zf.write(fp, os.path.basename(fp))
memory_zip.seek(0)

25
cps/dep_check.py
View File

@ -1,14 +1,16 @@
import os
import re
import sys
import json
from .constants import BASE_DIR
try:
from importlib_metadata import version
from importlib.metadata import version
importlib = True
ImportNotFound = BaseException
except ImportError:
importlib = False
version = None
if not importlib:
try:
@ -18,8 +20,16 @@ if not importlib:
except ImportError as e:
pkgresources = False
def load_dependencys(optional=False):
def load_dependencies(optional=False):
deps = list()
if getattr(sys, 'frozen', False):
pip_installed = os.path.join(BASE_DIR, ".pip_installed")
if os.path.exists(pip_installed):
with open(pip_installed) as f:
exe_deps = json.loads("".join(f.readlines()))
else:
return deps
if importlib or pkgresources:
if optional:
req_path = os.path.join(BASE_DIR, "optional-requirements.txt")
@ -31,11 +41,14 @@ def load_dependencys(optional=False):
if not line.startswith('#') and not line == '\n' and not line.startswith('git'):
res = re.match(r'(.*?)([<=>\s]+)([\d\.]+),?\s?([<=>\s]+)?([\d\.]+)?', line.strip())
try:
if getattr(sys, 'frozen', False):
dep_version = exe_deps[res.group(1).lower().replace('_', '-')]
else:
if importlib:
dep_version = version(res.group(1))
else:
dep_version = pkg_resources.get_distribution(res.group(1)).version
except ImportNotFound:
except (ImportNotFound, KeyError):
if optional:
continue
dep_version = "not installed"
@ -45,14 +58,14 @@ def load_dependencys(optional=False):
def dependency_check(optional=False):
d = list()
deps = load_dependencys(optional)
deps = load_dependencies(optional)
for dep in deps:
try:
dep_version_int = [int(x) for x in dep[0].split('.')]
low_check = [int(x) for x in dep[3].split('.')]
high_check = [int(x) for x in dep[5].split('.')]
except AttributeError:
high_check = None
high_check = []
except ValueError:
d.append({'name': dep[1],
'target': "available",

1739
cps/editbooks.py Normal file → Executable file
View File

File diff suppressed because it is too large Load Diff

127
cps/epub.py
View File

@ -20,24 +20,26 @@ import os
import zipfile
from lxml import etree
from . import isoLanguages
from . import isoLanguages, cover
from .helper import split_authors
from .constants import BookMeta
def extractCover(zipFile, coverFile, coverpath, tmp_file_name):
if coverFile is None:
def _extract_cover(zip_file, cover_file, cover_path, tmp_file_name):
if cover_file is None:
return None
else:
zipCoverPath = os.path.join(coverpath, coverFile).replace('\\', '/')
cf = zipFile.read(zipCoverPath)
cf = extension = None
zip_cover_path = os.path.join(cover_path, cover_file).replace('\\', '/')
prefix = os.path.splitext(tmp_file_name)[0]
tmp_cover_name = prefix + '.' + os.path.basename(zipCoverPath)
image = open(tmp_cover_name, 'wb')
image.write(cf)
image.close()
return tmp_cover_name
tmp_cover_name = prefix + '.' + os.path.basename(zip_cover_path)
ext = os.path.splitext(tmp_cover_name)
if len(ext) > 1:
extension = ext[1].lower()
if extension in cover.COVER_EXTENSIONS:
cf = zip_file.read(zip_cover_path)
return cover.cover_processing(tmp_file_name, cf, extension)
def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
@ -47,36 +49,44 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
'dc': 'http://purl.org/dc/elements/1.1/'
}
epubZip = zipfile.ZipFile(tmp_file_path)
epub_zip = zipfile.ZipFile(tmp_file_path)
txt = epubZip.read('META-INF/container.xml')
txt = epub_zip.read('META-INF/container.xml')
tree = etree.fromstring(txt)
cfname = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0]
cf = epubZip.read(cfname)
cf_name = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0]
cf = epub_zip.read(cf_name)
tree = etree.fromstring(cf)
coverpath = os.path.dirname(cfname)
cover_path = os.path.dirname(cf_name)
p = tree.xpath('/pkg:package/pkg:metadata', namespaces=ns)[0]
epub_metadata = {}
for s in ['title', 'description', 'creator', 'language', 'subject']:
for s in ['title', 'description', 'creator', 'language', 'subject', 'publisher', 'date']:
tmp = p.xpath('dc:%s/text()' % s, namespaces=ns)
if len(tmp) > 0:
if s == 'creator':
epub_metadata[s] = ' & '.join(split_authors(tmp))
elif s == 'subject':
epub_metadata[s] = ', '.join(tmp)
elif s == 'date':
epub_metadata[s] = tmp[0][:10]
else:
epub_metadata[s] = tmp[0]
else:
epub_metadata[s] = u'Unknown'
epub_metadata[s] = 'Unknown'
if epub_metadata['subject'] == u'Unknown':
if epub_metadata['subject'] == 'Unknown':
epub_metadata['subject'] = ''
if epub_metadata['description'] == u'Unknown':
if epub_metadata['publisher'] == 'Unknown':
epub_metadata['publisher'] = ''
if epub_metadata['date'] == 'Unknown':
epub_metadata['date'] = ''
if epub_metadata['description'] == 'Unknown':
description = tree.xpath("//*[local-name() = 'description']/text()")
if len(description) > 0:
epub_metadata['description'] = description
@ -86,9 +96,20 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
lang = epub_metadata['language'].split('-', 1)[0].lower()
epub_metadata['language'] = isoLanguages.get_lang3(lang)
epub_metadata = parse_epbub_series(ns, tree, epub_metadata)
epub_metadata = parse_epub_series(ns, tree, epub_metadata)
coverfile = parse_ebpub_cover(ns, tree, epubZip, coverpath, tmp_file_path)
cover_file = parse_epub_cover(ns, tree, epub_zip, cover_path, tmp_file_path)
identifiers = []
for node in p.xpath('dc:identifier', namespaces=ns):
try:
identifier_name = node.attrib.values()[-1]
except IndexError:
continue
identifier_value = node.text
if identifier_name in ('uuid', 'calibre') or identifier_value is None:
continue
identifiers.append([identifier_name, identifier_value])
if not epub_metadata['title']:
title = original_file_name
@ -100,45 +121,57 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
extension=original_file_extension,
title=title.encode('utf-8').decode('utf-8'),
author=epub_metadata['creator'].encode('utf-8').decode('utf-8'),
cover=coverfile,
cover=cover_file,
description=epub_metadata['description'],
tags=epub_metadata['subject'].encode('utf-8').decode('utf-8'),
series=epub_metadata['series'].encode('utf-8').decode('utf-8'),
series_id=epub_metadata['series_id'].encode('utf-8').decode('utf-8'),
languages=epub_metadata['language'],
publisher="")
publisher=epub_metadata['publisher'].encode('utf-8').decode('utf-8'),
pubdate=epub_metadata['date'],
identifiers=identifiers)
def parse_ebpub_cover(ns, tree, epubZip, coverpath, tmp_file_path):
coversection = tree.xpath("/pkg:package/pkg:manifest/pkg:item[@id='cover-image']/@href", namespaces=ns)
coverfile = None
if len(coversection) > 0:
coverfile = extractCover(epubZip, coversection[0], coverpath, tmp_file_path)
else:
def parse_epub_cover(ns, tree, epub_zip, cover_path, tmp_file_path):
cover_section = tree.xpath("/pkg:package/pkg:manifest/pkg:item[@id='cover-image']/@href", namespaces=ns)
cover_file = None
# if len(cover_section) > 0:
for cs in cover_section:
cover_file = _extract_cover(epub_zip, cs, cover_path, tmp_file_path)
if cover_file:
break
if not cover_file:
meta_cover = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='cover']/@content", namespaces=ns)
if len(meta_cover) > 0:
coversection = tree.xpath("/pkg:package/pkg:manifest/pkg:item[@id='"+meta_cover[0]+"']/@href", namespaces=ns)
cover_section = tree.xpath(
"/pkg:package/pkg:manifest/pkg:item[@id='"+meta_cover[0]+"']/@href", namespaces=ns)
if not cover_section:
cover_section = tree.xpath(
"/pkg:package/pkg:manifest/pkg:item[@properties='" + meta_cover[0] + "']/@href", namespaces=ns)
else:
coversection = tree.xpath("/pkg:package/pkg:guide/pkg:reference/@href", namespaces=ns)
if len(coversection) > 0:
filetype = coversection[0].rsplit('.', 1)[-1]
cover_section = tree.xpath("/pkg:package/pkg:guide/pkg:reference/@href", namespaces=ns)
for cs in cover_section:
filetype = cs.rsplit('.', 1)[-1]
if filetype == "xhtml" or filetype == "html": # if cover is (x)html format
markup = epubZip.read(os.path.join(coverpath, coversection[0]))
markupTree = etree.fromstring(markup)
markup = epub_zip.read(os.path.join(cover_path, cs))
markup_tree = etree.fromstring(markup)
# no matter xhtml or html with no namespace
imgsrc = markupTree.xpath("//*[local-name() = 'img']/@src")
img_src = markup_tree.xpath("//*[local-name() = 'img']/@src")
# Alternative image source
if not len(imgsrc):
imgsrc = markupTree.xpath("//attribute::*[contains(local-name(), 'href')]")
if len(imgsrc):
# imgsrc maybe startwith "../"" so fullpath join then relpath to cwd
filename = os.path.relpath(os.path.join(os.path.dirname(os.path.join(coverpath, coversection[0])),
imgsrc[0]))
coverfile = extractCover(epubZip, filename, "", tmp_file_path)
if not len(img_src):
img_src = markup_tree.xpath("//attribute::*[contains(local-name(), 'href')]")
if len(img_src):
# img_src maybe start with "../"" so fullpath join then relpath to cwd
filename = os.path.relpath(os.path.join(os.path.dirname(os.path.join(cover_path, cover_section[0])),
img_src[0]))
cover_file = _extract_cover(epub_zip, filename, "", tmp_file_path)
else:
coverfile = extractCover(epubZip, coversection[0], coverpath, tmp_file_path)
return coverfile
cover_file = _extract_cover(epub_zip, cs, cover_path, tmp_file_path)
if cover_file: break
return cover_file
def parse_epbub_series(ns, tree, epub_metadata):
def parse_epub_series(ns, tree, epub_metadata):
series = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='calibre:series']/@content", namespaces=ns)
if len(series) > 0:
epub_metadata['series'] = series[0]

6
cps/error_handler.py
View File

@ -17,6 +17,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import traceback
from flask import render_template
from werkzeug.exceptions import default_exceptions
try:
@ -42,8 +43,9 @@ def error_http(error):
def internal_error(error):
return render_template('http_error.html',
error_code="Internal Server Error",
error_name=str(error),
error_code="500 Internal Server Error",
error_name='The server encountered an internal error and was unable to complete your '
'request. There is an error in the application.',
issue=True,
unconfigured=False,
error_stack=traceback.format_exc().split("\n"),

18
cps/fb2.py
View File

@ -38,19 +38,19 @@ def get_fb2_info(tmp_file_path, original_file_extension):
if len(last_name):
last_name = last_name[0]
else:
last_name = u''
last_name = ''
middle_name = element.xpath('fb:middle-name/text()', namespaces=ns)
if len(middle_name):
middle_name = middle_name[0]
else:
middle_name = u''
middle_name = ''
first_name = element.xpath('fb:first-name/text()', namespaces=ns)
if len(first_name):
first_name = first_name[0]
else:
first_name = u''
return (first_name + u' '
+ middle_name + u' '
first_name = ''
return (first_name + ' '
+ middle_name + ' '
+ last_name)
author = str(", ".join(map(get_author, authors)))
@ -59,12 +59,12 @@ def get_fb2_info(tmp_file_path, original_file_extension):
if len(title):
title = str(title[0])
else:
title = u''
title = ''
description = tree.xpath('/fb:FictionBook/fb:description/fb:publish-info/fb:book-name/text()', namespaces=ns)
if len(description):
description = str(description[0])
else:
description = u''
description = ''
return BookMeta(
file_path=tmp_file_path,
@ -77,4 +77,6 @@ def get_fb2_info(tmp_file_path, original_file_extension):
series="",
series_id="",
languages="",
publisher="")
publisher="",
pubdate="",
identifiers=[])

95
cps/fs.py Normal file
View File

@ -0,0 +1,95 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2020 mmonkey
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from . import logger
from .constants import CACHE_DIR
from os import makedirs, remove
from os.path import isdir, isfile, join
from shutil import rmtree
class FileSystem:
_instance = None
_cache_dir = CACHE_DIR
def __new__(cls):
if cls._instance is None:
cls._instance = super(FileSystem, cls).__new__(cls)
cls.log = logger.create()
return cls._instance
def get_cache_dir(self, cache_type=None):
if not isdir(self._cache_dir):
try:
makedirs(self._cache_dir)
except OSError:
self.log.info(f'Failed to create path {self._cache_dir} (Permission denied).')
raise
path = join(self._cache_dir, cache_type)
if cache_type and not isdir(path):
try:
makedirs(path)
except OSError:
self.log.info(f'Failed to create path {path} (Permission denied).')
raise
return path if cache_type else self._cache_dir
def get_cache_file_dir(self, filename, cache_type=None):
path = join(self.get_cache_dir(cache_type), filename[:2])
if not isdir(path):
try:
makedirs(path)
except OSError:
self.log.info(f'Failed to create path {path} (Permission denied).')
raise
return path
def get_cache_file_path(self, filename, cache_type=None):
return join(self.get_cache_file_dir(filename, cache_type), filename) if filename else None
def get_cache_file_exists(self, filename, cache_type=None):
path = self.get_cache_file_path(filename, cache_type)
return isfile(path)
def delete_cache_dir(self, cache_type=None):
if not cache_type and isdir(self._cache_dir):
try:
rmtree(self._cache_dir)
except OSError:
self.log.info(f'Failed to delete path {self._cache_dir} (Permission denied).')
raise
path = join(self._cache_dir, cache_type)
if cache_type and isdir(path):
try:
rmtree(path)
except OSError:
self.log.info(f'Failed to delete path {path} (Permission denied).')
raise
def delete_cache_file(self, filename, cache_type=None):
path = self.get_cache_file_path(filename, cache_type)
if isfile(path):
try:
remove(path)
except OSError:
self.log.info(f'Failed to delete path {path} (Permission denied).')
raise

12
cps/gdrive.py
View File

@ -55,7 +55,7 @@ def authenticate_google_drive():
try:
authUrl = gdriveutils.Gauth.Instance().auth.GetAuthUrl()
except gdriveutils.InvalidConfigError:
flash(_(u'Google Drive setup not completed, try to deactivate and activate Google Drive again'),
flash(_('Google Drive setup not completed, try to deactivate and activate Google Drive again'),
category="error")
return redirect(url_for('web.index'))
return redirect(authUrl)
@ -91,9 +91,9 @@ def watch_gdrive():
config.save()
except HttpError as e:
reason=json.loads(e.content)['error']['errors'][0]
if reason['reason'] == u'push.webhookUrlUnauthorized':
flash(_(u'Callback domain is not verified, '
u'please follow steps to verify domain in google developer console'), category="error")
if reason['reason'] == 'push.webhookUrlUnauthorized':
flash(_('Callback domain is not verified, '
'please follow steps to verify domain in google developer console'), category="error")
else:
flash(reason['message'], category="error")
@ -109,7 +109,7 @@ def revoke_watch_gdrive():
try:
gdriveutils.stopChannel(gdriveutils.Gdrive.Instance().drive, last_watch_response['id'],
last_watch_response['resourceId'])
except HttpError:
except (HttpError, AttributeError):
pass
config.config_google_drive_watch_changes_response = {}
config.save()
@ -152,7 +152,7 @@ try:
move(os.path.join(tmp_dir, "tmp_metadata.db"), dbpath)
calibre_db.reconnect_db(config, ub.app_DB_path)
except Exception as ex:
log.debug_or_exception(ex)
log.error_or_exception(ex)
return ''
except AttributeError:
pass

116
cps/gdriveutils.py
View File

@ -32,13 +32,10 @@ try:
from sqlalchemy.orm import declarative_base
except ImportError:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.exc import OperationalError, InvalidRequestError
from sqlalchemy.exc import OperationalError, InvalidRequestError, IntegrityError
from sqlalchemy.orm.exc import StaleDataError
from sqlalchemy.sql.expression import text
try:
from six import __version__ as six_version
except ImportError:
six_version = "not installed"
try:
from httplib2 import __version__ as httplib2_version
except ImportError:
@ -56,16 +53,18 @@ try:
from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive
from pydrive2.auth import RefreshError
from pydrive2.files import ApiRequestError
except ImportError as err:
try:
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from pydrive.auth import RefreshError
from pydrive.files import ApiRequestError
except ImportError as err:
importError = err
gdrive_support = False
from . import logger, cli, config
from . import logger, cli_param, config
from .constants import CONFIG_DIR as _CONFIG_DIR
@ -79,7 +78,7 @@ if gdrive_support:
if not logger.is_debug_enabled():
logger.get('googleapiclient.discovery').setLevel(logger.logging.ERROR)
else:
log.debug("Cannot import pydrive,httplib2, using gdrive will not work: %s", importError)
log.debug("Cannot import pydrive, httplib2, using gdrive will not work: {}".format(importError))
class Singleton:
@ -139,11 +138,12 @@ class Gdrive:
def __init__(self):
self.drive = getDrive(gauth=Gauth.Instance().auth)
def is_gdrive_ready():
return os.path.exists(SETTINGS_YAML) and os.path.exists(CREDENTIALS)
engine = create_engine('sqlite:///{0}'.format(cli.gdpath), echo=False)
engine = create_engine('sqlite:///{0}'.format(cli_param.gd_path), echo=False)
Base = declarative_base()
# Open session for database connection
@ -191,10 +191,11 @@ def migrate():
session.execute('ALTER TABLE gdrive_ids2 RENAME to gdrive_ids')
break
if not os.path.exists(cli.gdpath):
if not os.path.exists(cli_param.gd_path):
try:
Base.metadata.create_all(engine)
except Exception:
except Exception as ex:
log.error("Error connect to database: {} - {}".format(cli_param.gd_path, ex))
raise
migrate()
@ -210,9 +211,9 @@ def getDrive(drive=None, gauth=None):
try:
gauth.Refresh()
except RefreshError as e:
log.error("Google Drive error: %s", e)
log.error("Google Drive error: {}".format(e))
except Exception as ex:
log.debug_or_exception(ex)
log.error_or_exception(ex)
else:
# Initialize the saved creds
gauth.Authorize()
@ -222,7 +223,7 @@ def getDrive(drive=None, gauth=None):
try:
drive.auth.Refresh()
except RefreshError as e:
log.error("Google Drive error: %s", e)
log.error("Google Drive error: {}".format(e))
return drive
def listRootFolders():
@ -231,7 +232,7 @@ def listRootFolders():
folder = "'root' in parents and mimeType = 'application/vnd.google-apps.folder' and trashed = false"
fileList = drive.ListFile({'q': folder}).GetList()
except (ServerNotFoundError, ssl.SSLError, RefreshError) as e:
log.info("GDrive Error %s" % e)
log.info("GDrive Error {}".format(e))
fileList = []
return fileList
@ -269,8 +270,7 @@ def getEbooksFolderId(drive=None):
try:
session.commit()
except OperationalError as ex:
log.error("gdrive.db DB is not Writeable")
log.debug('Database error: %s', ex)
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
return gDriveId.gdrive_id
@ -286,6 +286,7 @@ def getFile(pathId, fileName, drive):
def getFolderId(path, drive):
# drive = getDrive(drive)
currentFolderId = None
try:
currentFolderId = getEbooksFolderId(drive)
sqlCheckPath = path if path[-1] == '/' else path + '/'
@ -318,10 +319,14 @@ def getFolderId(path, drive):
session.commit()
else:
currentFolderId = storedPathName.gdrive_id
except OperationalError as ex:
log.error("gdrive.db DB is not Writeable")
log.debug('Database error: %s', ex)
except (OperationalError, IntegrityError, StaleDataError) as ex:
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
except ApiRequestError as ex:
log.error('{} {}'.format(ex.error['message'], path))
session.rollback()
except RefreshError as ex:
log.error(ex)
return currentFolderId
@ -355,7 +360,6 @@ def moveGdriveFolderRemote(origin_file, target_folder):
children = drive.auth.service.children().list(folderId=previous_parents).execute()
gFileTargetDir = getFileFromEbooksFolder(None, target_folder)
if not gFileTargetDir:
# Folder is not existing, create, and move folder
gFileTargetDir = drive.CreateFile(
{'title': target_folder, 'parents': [{"kind": "drive#fileLink", 'id': getEbooksFolderId()}],
"mimeType": "application/vnd.google-apps.folder"})
@ -365,6 +369,18 @@ def moveGdriveFolderRemote(origin_file, target_folder):
addParents=gFileTargetDir['id'],
removeParents=previous_parents,
fields='id, parents').execute()
elif gFileTargetDir['title'] != target_folder:
# Folder is not existing, create, and move folder
drive.auth.service.files().patch(fileId=origin_file['id'],
body={'title': target_folder},
fields='title').execute()
else:
# Move the file to the new folder
drive.auth.service.files().update(fileId=origin_file['id'],
addParents=gFileTargetDir['id'],
removeParents=previous_parents,
fields='id, parents').execute()
# if previous_parents has no children anymore, delete original fileparent
if len(children['items']) == 1:
deleteDatabaseEntry(previous_parents)
@ -412,24 +428,24 @@ def uploadFileToEbooksFolder(destFile, f):
splitDir = destFile.split('/')
for i, x in enumerate(splitDir):
if i == len(splitDir)-1:
existingFiles = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" %
existing_Files = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" %
(x.replace("'", r"\'"), parent['id'])}).GetList()
if len(existingFiles) > 0:
driveFile = existingFiles[0]
if len(existing_Files) > 0:
driveFile = existing_Files[0]
else:
driveFile = drive.CreateFile({'title': x,
'parents': [{"kind": "drive#fileLink", 'id': parent['id']}], })
driveFile.SetContentFile(f)
driveFile.Upload()
else:
existingFolder = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" %
existing_Folder = drive.ListFile({'q': "title = '%s' and '%s' in parents and trashed = false" %
(x.replace("'", r"\'"), parent['id'])}).GetList()
if len(existingFolder) == 0:
if len(existing_Folder) == 0:
parent = drive.CreateFile({'title': x, 'parents': [{"kind": "drive#fileLink", 'id': parent['id']}],
"mimeType": "application/vnd.google-apps.folder"})
parent.Upload()
else:
parent = existingFolder[0]
parent = existing_Folder[0]
def watchChange(drive, channel_id, channel_type, channel_address,
@ -528,8 +544,8 @@ def deleteDatabaseOnChange():
session.commit()
except (OperationalError, InvalidRequestError) as ex:
session.rollback()
log.debug('Database error: %s', ex)
log.error(u"GDrive DB is not Writeable")
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
def updateGdriveCalibreFromLocal():
@ -540,15 +556,14 @@ def updateGdriveCalibreFromLocal():
# update gdrive.db on edit of books title
def updateDatabaseOnEdit(ID,newPath):
sqlCheckPath = newPath if newPath[-1] == '/' else newPath + u'/'
sqlCheckPath = newPath if newPath[-1] == '/' else newPath + '/'
storedPathName = session.query(GdriveId).filter(GdriveId.gdrive_id == ID).first()
if storedPathName:
storedPathName.path = sqlCheckPath
try:
session.commit()
except OperationalError as ex:
log.error("gdrive.db DB is not Writeable")
log.debug('Database error: %s', ex)
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
@ -558,12 +573,12 @@ def deleteDatabaseEntry(ID):
try:
session.commit()
except OperationalError as ex:
log.error("gdrive.db DB is not Writeable")
log.debug('Database error: %s', ex)
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
# Gets cover file from gdrive
# ToDo: Check is this right everyone get read permissions on cover files?
def get_cover_via_gdrive(cover_path):
df = getFileFromEbooksFolder(cover_path, 'cover.jpg')
if df:
@ -580,8 +595,30 @@ def get_cover_via_gdrive(cover_path):
try:
session.commit()
except OperationalError as ex:
log.error("gdrive.db DB is not Writeable")
log.debug('Database error: %s', ex)
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
return df.metadata.get('webContentLink')
else:
return None
# Gets cover file from gdrive
def get_metadata_backup_via_gdrive(metadata_path):
df = getFileFromEbooksFolder(metadata_path, 'metadata.opf')
if df:
if not session.query(PermissionAdded).filter(PermissionAdded.gdrive_id == df['id']).first():
df.GetPermissions()
df.InsertPermission({
'type': 'anyone',
'value': 'anyone',
'role': 'writer', # ToDo needs write access
'withLink': True})
permissionAdded = PermissionAdded()
permissionAdded.gdrive_id = df['id']
session.add(permissionAdded)
try:
session.commit()
except OperationalError as ex:
log.error_or_exception('Database error: {}'.format(ex))
session.rollback()
return df.metadata.get('webContentLink')
else:
@ -603,7 +640,7 @@ def do_gdrive_download(df, headers, convert_encoding=False):
def stream(convert_encoding):
for byte in s:
headers = {"Range": 'bytes=%s-%s' % (byte[0], byte[1])}
headers = {"Range": 'bytes={}-{}'.format(byte[0], byte[1])}
resp, content = df.auth.Get_Http_Object().request(download_url, headers=headers)
if resp.status == 206:
if convert_encoding:
@ -611,7 +648,7 @@ def do_gdrive_download(df, headers, convert_encoding=False):
content = content.decode(result['encoding']).encode('utf-8')
yield content
else:
log.warning('An error occurred: %s', resp)
log.warning('An error occurred: {}'.format(resp))
return
return Response(stream_with_context(stream(convert_encoding)), headers=headers)
@ -668,8 +705,3 @@ def get_error_text(client_secrets=None):
return 'Callback url (redirect url) is missing in client_secrets.json'
if client_secrets:
client_secrets.update(filedata['web'])
def get_versions():
return {'six': six_version,
'httplib2': httplib2_version}

29
cps/gevent_wsgi.py Normal file
View File

@ -0,0 +1,29 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from gevent.pywsgi import WSGIHandler
class MyWSGIHandler(WSGIHandler):
def get_environ(self):
env = super().get_environ()
path, __ = self.path.split('?', 1) if '?' in self.path else (self.path, '')
env['RAW_URI'] = path
return env

803
cps/helper.py
View File

File diff suppressed because it is too large Load Diff

3
cps/isoLanguages.py
View File

@ -49,7 +49,7 @@ except ImportError:
def get_language_names(locale):
return _LANGUAGE_NAMES.get(locale)
return _LANGUAGE_NAMES.get(str(locale))
def get_language_name(locale, lang_code):
@ -57,6 +57,7 @@ def get_language_name(locale, lang_code):
return get_language_names(locale)[lang_code]
except KeyError:
log.error('Missing translation for language name: {}'.format(lang_code))
return "Unknown"
def get_language_codes(locale, language_names, remainder=None):

1971
cps/iso_language_names.py
View File

File diff suppressed because it is too large Load Diff

53
cps/jinjia.py
View File

@ -22,17 +22,17 @@
# custom jinja filters
from markupsafe import escape
import datetime
import mimetypes
from uuid import uuid4
from babel.dates import format_date
# from babel.dates import format_date
from flask import Blueprint, request, url_for
from flask_babel import get_locale
from flask_babel import format_date
from flask_login import current_user
from markupsafe import escape
from . import logger
from . import constants, logger
jinjia = Blueprint('jinjia', __name__)
log = logger.create()
@ -77,7 +77,7 @@ def mimetype_filter(val):
@jinjia.app_template_filter('formatdate')
def formatdate_filter(val):
try:
return format_date(val, format='medium', locale=get_locale())
return format_date(val, format='medium')
except AttributeError as e:
log.error('Babel error: %s, Current user locale: %s, Current User: %s', e,
current_user.locale,
@ -128,12 +128,55 @@ def formatseriesindex_filter(series_index):
return series_index
return 0
@jinjia.app_template_filter('escapedlink')
def escapedlink_filter(url, text):
return "<a href='{}'>{}</a>".format(url, escape(text))
@jinjia.app_template_filter('uuidfilter')
def uuidfilter(var):
return uuid4()
@jinjia.app_template_filter('cache_timestamp')
def cache_timestamp(rolling_period='month'):
if rolling_period == 'day':
return str(int(datetime.datetime.today().replace(hour=1, minute=1).timestamp()))
elif rolling_period == 'year':
return str(int(datetime.datetime.today().replace(day=1).timestamp()))
else:
return str(int(datetime.datetime.today().replace(month=1, day=1).timestamp()))
@jinjia.app_template_filter('last_modified')
def book_last_modified(book):
return str(int(book.last_modified.timestamp()))
@jinjia.app_template_filter('get_cover_srcset')
def get_cover_srcset(book):
srcset = list()
resolutions = {
constants.COVER_THUMBNAIL_SMALL: 'sm',
constants.COVER_THUMBNAIL_MEDIUM: 'md',
constants.COVER_THUMBNAIL_LARGE: 'lg'
}
for resolution, shortname in resolutions.items():
url = url_for('web.get_cover', book_id=book.id, resolution=shortname, c=book_last_modified(book))
srcset.append(f'{url} {resolution}x')
return ', '.join(srcset)
@jinjia.app_template_filter('get_series_srcset')
def get_cover_srcset(series):
srcset = list()
resolutions = {
constants.COVER_THUMBNAIL_SMALL: 'sm',
constants.COVER_THUMBNAIL_MEDIUM: 'md',
constants.COVER_THUMBNAIL_LARGE: 'lg'
}
for resolution, shortname in resolutions.items():
url = url_for('web.get_series_cover', series_id=series.id, resolution=shortname, c=cache_timestamp())
srcset.append(f'{url} {resolution}x')
return ', '.join(srcset)

206
cps/kobo.py
View File

@ -22,11 +22,8 @@ import datetime
import os
import uuid
from time import gmtime, strftime
try:
from urllib import unquote
except ImportError:
from urllib.parse import unquote
import json
from urllib.parse import unquote
from flask import (
Blueprint,
@ -48,7 +45,8 @@ import requests
from . import config, logger, kobo_auth, db, calibre_db, helper, shelf as shelf_lib, ub, csrf, kobo_sync_status
from .constants import sqlalchemy_version2
from . import isoLanguages
from .constants import sqlalchemy_version2, COVER_THUMBNAIL_SMALL
from .helper import get_download_link
from .services import SyncToken as SyncToken
from .web import download_required
@ -102,6 +100,8 @@ def make_request_to_kobo_store(sync_token=None):
allow_redirects=False,
timeout=(2, 10)
)
log.debug("Content: " + str(store_response.content))
log.debug("StatusCode: " + str(store_response.status_code))
return store_response
@ -110,7 +110,8 @@ def redirect_or_proxy_request():
if request.method == "GET":
return redirect(get_store_url_for_current_request(), 307)
else:
# The Kobo device turns other request types into GET requests on redirects, so we instead proxy to the Kobo store ourselves.
# The Kobo device turns other request types into GET requests on redirects,
# so we instead proxy to the Kobo store ourselves.
store_response = make_request_to_kobo_store()
response_headers = store_response.headers
@ -129,7 +130,7 @@ def convert_to_kobo_timestamp_string(timestamp):
return timestamp.strftime("%Y-%m-%dT%H:%M:%SZ")
except AttributeError as exc:
log.debug("Timestamp not valid: {}".format(exc))
return datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
return datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
@kobo.route("/v1/library/sync")
@ -142,16 +143,20 @@ def HandleSyncRequest():
if not current_app.wsgi_app.is_proxied:
log.debug('Kobo: Received unproxied request, changed request port to external server port')
# TODO: Limit the number of books return per sync call, and rely on the sync-continuatation header
# instead so that the device triggers another sync.
# if no books synced don't respect sync_token
if not ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.user_id == current_user.id).count():
sync_token.books_last_modified = datetime.datetime.min
sync_token.books_last_created = datetime.datetime.min
sync_token.reading_state_last_modified = datetime.datetime.min
new_books_last_modified = sync_token.books_last_modified
new_books_last_created = sync_token.books_last_created
new_books_last_modified = sync_token.books_last_modified # needed for sync selected shelfs only
new_books_last_created = sync_token.books_last_created # needed to distinguish between new and changed entitlement
new_reading_state_last_modified = sync_token.reading_state_last_modified
new_archived_last_modified = datetime.datetime.min
sync_results = []
# We reload the book database so that the user get's a fresh view of the library
# We reload the book database so that the user gets a fresh view of the library
# in case of external changes (e.g: adding a book through Calibre).
calibre_db.reconnect_db(config, ub.app_DB_path)
@ -169,10 +174,10 @@ def HandleSyncRequest():
ub.BookShelf.date_added,
ub.ArchivedBook.is_archived)
changed_entries = (changed_entries
.join(db.Data).outerjoin(ub.ArchivedBook, db.Books.id == ub.ArchivedBook.book_id)
.join(ub.KoboSyncedBooks, ub.KoboSyncedBooks.book_id == db.Books.id, isouter=True)
.filter(or_(ub.KoboSyncedBooks.user_id != current_user.id,
ub.KoboSyncedBooks.book_id == None))
.join(db.Data).outerjoin(ub.ArchivedBook, and_(db.Books.id == ub.ArchivedBook.book_id,
ub.ArchivedBook.user_id == current_user.id))
.filter(db.Books.id.notin_(calibre_db.session.query(ub.KoboSyncedBooks.book_id)
.filter(ub.KoboSyncedBooks.user_id == current_user.id)))
.filter(ub.BookShelf.date_added > sync_token.books_last_modified)
.filter(db.Data.format.in_(KOBO_FORMATS))
.filter(calibre_db.common_filters(allow_show_archived=True))
@ -182,8 +187,7 @@ def HandleSyncRequest():
.join(ub.Shelf)
.filter(ub.Shelf.user_id == current_user.id)
.filter(ub.Shelf.kobo_sync)
.distinct()
)
.distinct())
else:
if sqlalchemy_version2:
changed_entries = select(db.Books, ub.ArchivedBook.last_modified, ub.ArchivedBook.is_archived)
@ -192,26 +196,24 @@ def HandleSyncRequest():
ub.ArchivedBook.last_modified,
ub.ArchivedBook.is_archived)
changed_entries = (changed_entries
.join(db.Data).outerjoin(ub.ArchivedBook, db.Books.id == ub.ArchivedBook.book_id)
.join(ub.KoboSyncedBooks, ub.KoboSyncedBooks.book_id == db.Books.id, isouter=True)
.filter(or_(ub.KoboSyncedBooks.user_id != current_user.id,
ub.KoboSyncedBooks.book_id == None))
.filter(calibre_db.common_filters())
.join(db.Data).outerjoin(ub.ArchivedBook, and_(db.Books.id == ub.ArchivedBook.book_id,
ub.ArchivedBook.user_id == current_user.id))
.filter(db.Books.id.notin_(calibre_db.session.query(ub.KoboSyncedBooks.book_id)
.filter(ub.KoboSyncedBooks.user_id == current_user.id)))
.filter(calibre_db.common_filters(allow_show_archived=True))
.filter(db.Data.format.in_(KOBO_FORMATS))
.order_by(db.Books.last_modified)
.order_by(db.Books.id)
)
.order_by(db.Books.id))
reading_states_in_new_entitlements = []
if sqlalchemy_version2:
books = calibre_db.session.execute(changed_entries.limit(SYNC_ITEM_LIMIT))
else:
books = changed_entries.limit(SYNC_ITEM_LIMIT)
log.debug("Books to Sync: {}".format(len(books.all())))
for book in books:
kobo_sync_status.add_synced_books(book.Books.id)
formats = [data.format for data in book.Books.data]
if not 'KEPUB' in formats and config.config_kepubifypath and 'EPUB' in formats:
if 'KEPUB' not in formats and config.config_kepubifypath and 'EPUB' in formats:
helper.convert_book_format(book.Books.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
kobo_reading_state = get_or_create_reading_state(book.Books.id)
@ -248,14 +250,17 @@ def HandleSyncRequest():
pass
new_books_last_created = max(ts_created, new_books_last_created)
kobo_sync_status.add_synced_books(book.Books.id)
if sqlalchemy_version2:
max_change = calibre_db.session.execute(changed_entries
.filter(ub.ArchivedBook.is_archived)
.filter(ub.ArchivedBook.user_id == current_user.id)
.order_by(func.datetime(ub.ArchivedBook.last_modified).desc()))\
.columns(db.Books).first()
else:
max_change = changed_entries.from_self().filter(ub.ArchivedBook.is_archived) \
max_change = changed_entries.from_self().filter(ub.ArchivedBook.is_archived)\
.filter(ub.ArchivedBook.user_id == current_user.id) \
.order_by(func.datetime(ub.ArchivedBook.last_modified).desc()).first()
max_change = max_change.last_modified if max_change else new_archived_last_modified
@ -290,7 +295,8 @@ def HandleSyncRequest():
changed_reading_states = changed_reading_states.filter(
and_(ub.KoboReadingState.user_id == current_user.id,
ub.KoboReadingState.book_id.notin_(reading_states_in_new_entitlements)))
ub.KoboReadingState.book_id.notin_(reading_states_in_new_entitlements)))\
.order_by(ub.KoboReadingState.last_modified)
cont_sync |= bool(changed_reading_states.count() > SYNC_ITEM_LIMIT)
for kobo_reading_state in changed_reading_states.limit(SYNC_ITEM_LIMIT).all():
book = calibre_db.session.query(db.Books).filter(db.Books.id == kobo_reading_state.book_id).one_or_none()
@ -304,18 +310,19 @@ def HandleSyncRequest():
sync_shelves(sync_token, sync_results, only_kobo_shelves)
# update last created timestamp to distinguish between new and changed entitlements
if not cont_sync:
sync_token.books_last_created = new_books_last_created
sync_token.books_last_modified = new_books_last_modified
sync_token.archive_last_modified = new_archived_last_modified
sync_token.reading_state_last_modified = new_reading_state_last_modified
# sync_token.books_last_id = books_last_id
return generate_sync_response(sync_token, sync_results, cont_sync)
def generate_sync_response(sync_token, sync_results, set_cont=False):
extra_headers = {}
if config.config_kobo_proxy:
if config.config_kobo_proxy and not set_cont:
# Merge in sync results from the official Kobo store.
try:
store_response = make_request_to_kobo_store(sync_token)
@ -334,8 +341,9 @@ def generate_sync_response(sync_token, sync_results, set_cont=False):
sync_token.to_headers(extra_headers)
# log.debug("Kobo Sync Content: {}".format(sync_results))
response = make_response(jsonify(sync_results), extra_headers)
# jsonify decodes the unicode string different to what kobo expects
response = make_response(json.dumps(sync_results), extra_headers)
response.headers["Content-Type"] = "application/json; charset=utf-8"
return response
@ -348,11 +356,13 @@ def HandleMetadataRequest(book_uuid):
log.info("Kobo library metadata request received for book %s" % book_uuid)
book = calibre_db.get_book_by_uuid(book_uuid)
if not book or not book.data:
log.info(u"Book %s not found in database", book_uuid)
log.info("Book %s not found in database", book_uuid)
return redirect_or_proxy_request()
metadata = get_metadata(book)
return jsonify([metadata])
response = make_response(json.dumps([metadata], ensure_ascii=False))
response.headers["Content-Type"] = "application/json; charset=utf-8"
return response
def get_download_url_for_book(book, book_format):
@ -380,10 +390,10 @@ def get_download_url_for_book(book, book_format):
def create_book_entitlement(book, archived):
book_uuid = book.uuid
book_uuid = str(book.uuid)
return {
"Accessibility": "Full",
"ActivePeriod": {"From": convert_to_kobo_timestamp_string(datetime.datetime.now())},
"ActivePeriod": {"From": convert_to_kobo_timestamp_string(datetime.datetime.utcnow())},
"Created": convert_to_kobo_timestamp_string(book.timestamp),
"CrossRevisionId": book_uuid,
"Id": book_uuid,
@ -407,18 +417,15 @@ def get_description(book):
return book.comments[0].text
# TODO handle multiple authors
def get_author(book):
if not book.authors:
return {"Contributors": None}
if len(book.authors) > 1:
author_list = []
autor_roles = []
for author in book.authors:
autor_roles.append({"Name":author.name, "Role":"Author"})
autor_roles.append({"Name": author.name})
author_list.append(author.name)
return {"ContributorRoles": autor_roles, "Contributors":author_list}
return {"ContributorRoles": [{"Name":book.authors[0].name, "Role":"Author"}], "Contributors": book.authors[0].name}
return {"ContributorRoles": autor_roles, "Contributors": author_list}
def get_publisher(book):
@ -432,10 +439,17 @@ def get_series(book):
return None
return book.series[0].name
def get_seriesindex(book):
return book.series_index or 1
def get_language(book):
if not book.languages:
return 'en'
return isoLanguages.get(part3=book.languages[0].lang_code).part1
def get_metadata(book):
download_urls = []
kepub = [data for data in book.data if data.format == 'KEPUB']
@ -473,12 +487,10 @@ def get_metadata(book):
"IsInternetArchive": False,
"IsPreOrder": False,
"IsSocialEnabled": True,
"Language": "en",
"Language": get_language(book),
"PhoneticPronunciations": {},
# TODO: Fix book.pubdate to return a datetime object so that we can easily
# convert it to the format Kobo devices expect.
"PublicationDate": book.pubdate,
"Publisher": {"Imprint": "", "Name": get_publisher(book),},
"PublicationDate": convert_to_kobo_timestamp_string(book.pubdate),
"Publisher": {"Imprint": "", "Name": get_publisher(book), },
"RevisionId": book_uuid,
"Title": book.title,
"WorkId": book_uuid,
@ -492,17 +504,18 @@ def get_metadata(book):
"Number": get_seriesindex(book), # ToDo Check int() ?
"NumberFloat": float(get_seriesindex(book)),
# Get a deterministic id based on the series name.
"Id": uuid.uuid3(uuid.NAMESPACE_DNS, name),
"Id": str(uuid.uuid3(uuid.NAMESPACE_DNS, name)),
}
return metadata
@csrf.exempt
@kobo.route("/v1/library/tags", methods=["POST", "DELETE"])
@requires_kobo_auth
# Creates a Shelf with the given items, and returns the shelf's uuid.
def HandleTagCreate():
# catch delete requests, otherwise the are handeld in the book delete handler
# catch delete requests, otherwise the are handled in the book delete handler
if request.method == "DELETE":
abort(405)
name, items = None, None
@ -545,11 +558,9 @@ def HandleTagUpdate(tag_id):
else:
abort(404, description="Collection isn't known to CalibreWeb")
if not shelf_lib.check_shelf_edit_permissions(shelf):
abort(401, description="User is unauthaurized to edit shelf.")
if request.method == "DELETE":
shelf_lib.delete_shelf_helper(shelf)
if not shelf_lib.delete_shelf_helper(shelf):
abort(401, description="Error deleting Shelf")
else:
name = None
try:
@ -667,11 +678,8 @@ def HandleTagRemoveItem(tag_id):
# Note: Public shelves that aren't owned by the user aren't supported.
def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
new_tags_last_modified = sync_token.tags_last_modified
for shelf in ub.session.query(ub.ShelfArchive).filter(
func.datetime(ub.ShelfArchive.last_modified) > sync_token.tags_last_modified,
ub.ShelfArchive.user_id == current_user.id
):
# transmit all archived shelfs independent of last sync (why should this matter?)
for shelf in ub.session.query(ub.ShelfArchive).filter(ub.ShelfArchive.user_id == current_user.id):
new_tags_last_modified = max(shelf.last_modified, new_tags_last_modified)
sync_results.append({
"DeletedTag": {
@ -684,7 +692,6 @@ def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
ub.session.delete(shelf)
ub.session_commit()
extra_filters = []
if only_kobo_shelves:
for shelf in ub.session.query(ub.Shelf).filter(
@ -717,7 +724,6 @@ def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
*extra_filters
).distinct().order_by(func.datetime(ub.Shelf.last_modified).asc())
for shelf in shelflist:
if not shelf_lib.check_shelf_view_permissions(shelf):
continue
@ -753,7 +759,7 @@ def create_kobo_tag(shelf):
for book_shelf in shelf.books:
book = calibre_db.get_book(book_shelf.book_id)
if not book:
log.info(u"Book (id: %s) in BookShelf (id: %s) not found in book database", book_shelf.book_id, shelf.id)
log.info("Book (id: %s) in BookShelf (id: %s) not found in book database", book_shelf.book_id, shelf.id)
continue
tag["Items"].append(
{
@ -763,13 +769,14 @@ def create_kobo_tag(shelf):
)
return {"Tag": tag}
@csrf.exempt
@kobo.route("/v1/library/<book_uuid>/state", methods=["GET", "PUT"])
@requires_kobo_auth
def HandleStateRequest(book_uuid):
book = calibre_db.get_book_by_uuid(book_uuid)
if not book or not book.data:
log.info(u"Book %s not found in database", book_uuid)
log.info("Book %s not found in database", book_uuid)
return redirect_or_proxy_request()
kobo_reading_state = get_or_create_reading_state(book.id)
@ -847,7 +854,7 @@ def get_ub_read_status(kobo_read_status):
def get_or_create_reading_state(book_id):
book_read = ub.session.query(ub.ReadBook).filter(ub.ReadBook.book_id == book_id,
ub.ReadBook.user_id == current_user.id).one_or_none()
ub.ReadBook.user_id == int(current_user.id)).one_or_none()
if not book_read:
book_read = ub.ReadBook(user_id=current_user.id, book_id=book_id)
if not book_read.kobo_reading_state:
@ -911,13 +918,12 @@ def get_current_bookmark_response(current_bookmark):
}
return resp
@kobo.route("/<book_uuid>/<width>/<height>/<isGreyscale>/image.jpg", defaults={'Quality': ""})
@kobo.route("/<book_uuid>/<width>/<height>/<Quality>/<isGreyscale>/image.jpg")
@requires_kobo_auth
def HandleCoverImageRequest(book_uuid, width, height,Quality, isGreyscale):
book_cover = helper.get_book_cover_with_uuid(
book_uuid, use_generic_cover_on_failure=False
)
def HandleCoverImageRequest(book_uuid, width, height, Quality, isGreyscale):
book_cover = helper.get_book_cover_with_uuid(book_uuid, resolution=COVER_THUMBNAIL_SMALL)
if not book_cover:
if config.config_kobo_proxy:
log.debug("Cover for unknown book: %s proxied to kobo" % book_uuid)
@ -942,25 +948,16 @@ def TopLevelEndpoint():
@kobo.route("/v1/library/<book_uuid>", methods=["DELETE"])
@requires_kobo_auth
def HandleBookDeletionRequest(book_uuid):
log.info("Kobo book deletion request received for book %s" % book_uuid)
log.info("Kobo book delete request received for book %s" % book_uuid)
book = calibre_db.get_book_by_uuid(book_uuid)
if not book:
log.info(u"Book %s not found in database", book_uuid)
log.info("Book %s not found in database", book_uuid)
return redirect_or_proxy_request()
book_id = book.id
archived_book = (
ub.session.query(ub.ArchivedBook)
.filter(ub.ArchivedBook.book_id == book_id)
.first()
)
if not archived_book:
archived_book = ub.ArchivedBook(user_id=current_user.id, book_id=book_id)
archived_book.is_archived = True
archived_book.last_modified = datetime.datetime.utcnow()
ub.session.merge(archived_book)
ub.session_commit()
is_archived = kobo_sync_status.change_archived_books(book_id, True)
if is_archived:
kobo_sync_status.remove_synced_book(book_id)
return "", 204
@ -968,7 +965,7 @@ def HandleBookDeletionRequest(book_uuid):
@csrf.exempt
@kobo.route("/v1/library/<dummy>", methods=["DELETE", "GET"])
def HandleUnimplementedRequest(dummy=None):
log.debug("Unimplemented Library Request received: %s", request.base_url)
log.debug("Unimplemented Library Request received: %s (request is forwarded to kobo if configured)", request.base_url)
return redirect_or_proxy_request()
@ -980,22 +977,47 @@ def HandleUnimplementedRequest(dummy=None):
@kobo.route("/v1/user/recommendations", methods=["GET", "POST"])
@kobo.route("/v1/analytics/<dummy>", methods=["GET", "POST"])
def HandleUserRequest(dummy=None):
log.debug("Unimplemented User Request received: %s", request.base_url)
log.debug("Unimplemented User Request received: %s (request is forwarded to kobo if configured)", request.base_url)
return redirect_or_proxy_request()
@csrf.exempt
@kobo.route("/v1/user/loyalty/benefits", methods=["GET"])
def handle_benefits():
if config.config_kobo_proxy:
return redirect_or_proxy_request()
else:
return make_response(jsonify({"Benefits": {}}))
@csrf.exempt
@kobo.route("/v1/analytics/gettests", methods=["GET", "POST"])
def handle_getests():
if config.config_kobo_proxy:
return redirect_or_proxy_request()
else:
testkey = request.headers.get("X-Kobo-userkey", "")
return make_response(jsonify({"Result": "Success", "TestKey": testkey, "Tests": {}}))
@csrf.exempt
@kobo.route("/v1/products/<dummy>/prices", methods=["GET", "POST"])
@kobo.route("/v1/products/<dummy>/recommendations", methods=["GET", "POST"])
@kobo.route("/v1/products/<dummy>/nextread", methods=["GET", "POST"])
@kobo.route("/v1/products/<dummy>/reviews", methods=["GET", "POST"])
@kobo.route("/v1/products/featured/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/featured/", methods=["GET", "POST"])
@kobo.route("/v1/products/books/external/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/books/series/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/books/<dummy>", methods=["GET", "POST"])
@kobo.route("/v1/products/books/<dummy>/", methods=["GET", "POST"])
@kobo.route("/v1/products/dailydeal", methods=["GET", "POST"])
@kobo.route("/v1/products/deals", methods=["GET", "POST"])
@kobo.route("/v1/products", methods=["GET", "POST"])
@kobo.route("/v1/affiliate", methods=["GET", "POST"])
@kobo.route("/v1/deals", methods=["GET", "POST"])
def HandleProductsRequest(dummy=None):
log.debug("Unimplemented Products Request received: %s", request.base_url)
log.debug("Unimplemented Products Request received: %s (request is forwarded to kobo if configured)", request.base_url)
return redirect_or_proxy_request()
@ -1143,14 +1165,16 @@ def NATIVE_KOBO_RESOURCES():
"eula_page": "https://www.kobo.com/termsofuse?style=onestore",
"exchange_auth": "https://storeapi.kobo.com/v1/auth/exchange",
"external_book": "https://storeapi.kobo.com/v1/products/books/external/{Ids}",
"facebook_sso_page": "https://authorize.kobo.com/signin/provider/Facebook/login?returnUrl=http://store.kobobooks.com/",
"facebook_sso_page":
"https://authorize.kobo.com/signin/provider/Facebook/login?returnUrl=http://store.kobobooks.com/",
"featured_list": "https://storeapi.kobo.com/v1/products/featured/{FeaturedListId}",
"featured_lists": "https://storeapi.kobo.com/v1/products/featured",
"free_books_page": {
"EN": "https://www.kobo.com/{region}/{language}/p/free-ebooks",
"FR": "https://www.kobo.com/{region}/{language}/p/livres-gratuits",
"IT": "https://www.kobo.com/{region}/{language}/p/libri-gratuiti",
"NL": "https://www.kobo.com/{region}/{language}/List/bekijk-het-overzicht-van-gratis-ebooks/QpkkVWnUw8sxmgjSlCbJRg",
"NL": "https://www.kobo.com/{region}/{language}/"
"List/bekijk-het-overzicht-van-gratis-ebooks/QpkkVWnUw8sxmgjSlCbJRg",
"PT": "https://www.kobo.com/{region}/{language}/p/livros-gratis",
},
"fte_feedback": "https://storeapi.kobo.com/v1/products/ftefeedback",
@ -1175,7 +1199,8 @@ def NATIVE_KOBO_RESOURCES():
"library_stack": "https://storeapi.kobo.com/v1/user/library/stacks/{LibraryItemId}",
"library_sync": "https://storeapi.kobo.com/v1/library/sync",
"love_dashboard_page": "https://store.kobobooks.com/{culture}/kobosuperpoints",
"love_points_redemption_page": "https://store.kobobooks.com/{culture}/KoboSuperPointsRedemption?productId={ProductId}",
"love_points_redemption_page":
"https://store.kobobooks.com/{culture}/KoboSuperPointsRedemption?productId={ProductId}",
"magazine_landing_page": "https://store.kobobooks.com/emagazines",
"notifications_registration_issue": "https://storeapi.kobo.com/v1/notifications/registration",
"oauth_host": "https://oauth.kobo.com",
@ -1191,7 +1216,8 @@ def NATIVE_KOBO_RESOURCES():
"product_recommendations": "https://storeapi.kobo.com/v1/products/{ProductId}/recommendations",
"product_reviews": "https://storeapi.kobo.com/v1/products/{ProductIds}/reviews",
"products": "https://storeapi.kobo.com/v1/products",
"provider_external_sign_in_page": "https://authorize.kobo.com/ExternalSignIn/{providerName}?returnUrl=http://store.kobobooks.com/",
"provider_external_sign_in_page":
"https://authorize.kobo.com/ExternalSignIn/{providerName}?returnUrl=http://store.kobobooks.com/",
"purchase_buy": "https://www.kobo.com/checkout/createpurchase/",
"purchase_buy_templated": "https://www.kobo.com/{culture}/checkout/createpurchase/{ProductId}",
"quickbuy_checkout": "https://storeapi.kobo.com/v1/store/quickbuy/{PurchaseId}/checkout",

132
cps/kobo_auth.py
View File

@ -62,6 +62,7 @@ particular calls to non-Kobo specific endpoints such as the CalibreWeb book down
from binascii import hexlify
from datetime import datetime
from os import urandom
from functools import wraps
from flask import g, Blueprint, url_for, abort, request
from flask_login import login_user, current_user, login_required
@ -70,20 +71,61 @@ from flask_babel import gettext as _
from . import logger, config, calibre_db, db, helper, ub, lm
from .render_template import render_title_template
try:
from functools import wraps
except ImportError:
pass # We're not using Python 3
log = logger.create()
kobo_auth = Blueprint("kobo_auth", __name__, url_prefix="/kobo_auth")
def register_url_value_preprocessor(kobo):
@kobo.url_value_preprocessor
# pylint: disable=unused-variable
def pop_auth_token(__, values):
g.auth_token = values.pop("auth_token")
@kobo_auth.route("/generate_auth_token/<int:user_id>")
@login_required
def generate_auth_token(user_id):
warning = False
host_list = request.host.rsplit(':')
if len(host_list) == 1:
host = ':'.join(host_list)
else:
host = ':'.join(host_list[0:-1])
if host.startswith('127.') or host.lower() == 'localhost' or host.startswith('[::ffff:7f') or host == "[::1]":
warning = _('Please access Calibre-Web from non localhost to get valid api_endpoint for kobo device')
# Generate auth token if none is existing for this user
auth_token = ub.session.query(ub.RemoteAuthToken).filter(
ub.RemoteAuthToken.user_id == user_id
).filter(ub.RemoteAuthToken.token_type==1).first()
if not auth_token:
auth_token = ub.RemoteAuthToken()
auth_token.user_id = user_id
auth_token.expiration = datetime.max
auth_token.auth_token = (hexlify(urandom(16))).decode("utf-8")
auth_token.token_type = 1
ub.session.add(auth_token)
ub.session_commit()
books = calibre_db.session.query(db.Books).join(db.Data).all()
for book in books:
formats = [data.format for data in book.data]
if 'KEPUB' not in formats and config.config_kepubifypath and 'EPUB' in formats:
helper.convert_book_format(book.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
return render_title_template(
"generate_kobo_auth_url.html",
title=_("Kobo Setup"),
auth_token=auth_token.auth_token,
warning = warning
)
@kobo_auth.route("/deleteauthtoken/<int:user_id>", methods=["POST"])
@login_required
def delete_auth_token(user_id):
# Invalidate any previously generated Kobo Auth token for this user
ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.user_id == user_id)\
.filter(ub.RemoteAuthToken.token_type==1).delete()
return ub.session_commit()
def disable_failed_auth_redirect_for_blueprint(bp):
@ -97,6 +139,13 @@ def get_auth_token():
return None
def register_url_value_preprocessor(kobo):
@kobo.url_value_preprocessor
# pylint: disable=unused-variable
def pop_auth_token(__, values):
g.auth_token = values.pop("auth_token")
def requires_kobo_auth(f):
@wraps(f)
def inner(*args, **kwargs):
@ -114,64 +163,3 @@ def requires_kobo_auth(f):
log.debug("Received Kobo request without a recognizable auth token.")
return abort(401)
return inner
kobo_auth = Blueprint("kobo_auth", __name__, url_prefix="/kobo_auth")
@kobo_auth.route("/generate_auth_token/<int:user_id>")
@login_required
def generate_auth_token(user_id):
host_list = request.host.rsplit(':')
if len(host_list) == 1:
host = ':'.join(host_list)
else:
host = ':'.join(host_list[0:-1])
if host.startswith('127.') or host.lower() == 'localhost' or host.startswith('[::ffff:7f'):
warning = _('PLease access calibre-web from non localhost to get valid api_endpoint for kobo device')
return render_title_template(
"generate_kobo_auth_url.html",
title=_(u"Kobo Setup"),
warning = warning
)
else:
# Invalidate any prevously generated Kobo Auth token for this user.
auth_token = ub.session.query(ub.RemoteAuthToken).filter(
ub.RemoteAuthToken.user_id == user_id
).filter(ub.RemoteAuthToken.token_type==1).first()
if not auth_token:
auth_token = ub.RemoteAuthToken()
auth_token.user_id = user_id
auth_token.expiration = datetime.max
auth_token.auth_token = (hexlify(urandom(16))).decode("utf-8")
auth_token.token_type = 1
ub.session.add(auth_token)
ub.session_commit()
books = calibre_db.session.query(db.Books).join(db.Data).all()
for book in books:
formats = [data.format for data in book.data]
if not 'KEPUB' in formats and config.config_kepubifypath and 'EPUB' in formats:
helper.convert_book_format(book.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
return render_title_template(
"generate_kobo_auth_url.html",
title=_(u"Kobo Setup"),
kobo_auth_url=url_for(
"kobo.TopLevelEndpoint", auth_token=auth_token.auth_token, _external=True
),
warning = False
)
@kobo_auth.route("/deleteauthtoken/<int:user_id>")
@login_required
def delete_auth_token(user_id):
# Invalidate any prevously generated Kobo Auth token for this user.
ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.user_id == user_id)\
.filter(ub.RemoteAuthToken.token_type==1).delete()
return ub.session_commit()

65
cps/kobo_sync_status.py
View File

@ -20,10 +20,15 @@
from flask_login import current_user
from . import ub
import datetime
from sqlalchemy.sql.expression import or_
from sqlalchemy.sql.expression import or_, and_, true
from sqlalchemy import exc
# Add the current book id to kobo_synced_books table for current user, if entry is already present,
# do nothing (safety precaution)
def add_synced_books(book_id):
is_present = ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id)\
.filter(ub.KoboSyncedBooks.user_id == current_user.id).count()
if not is_present:
synced_book = ub.KoboSyncedBooks()
synced_book.user_id = current_user.id
synced_book.book_id = book_id
@ -31,41 +36,53 @@ def add_synced_books(book_id):
ub.session_commit()
def remove_synced_book(book_id):
ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id).delete()
# Select all entries of current book in kobo_synced_books table, which are from current user and delete them
def remove_synced_book(book_id, all=False, session=None):
if not all:
user = ub.KoboSyncedBooks.user_id == current_user.id
else:
user = true()
if not session:
ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id).filter(user).delete()
ub.session_commit()
else:
session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == book_id).filter(user).delete()
ub.session_commit(_session=session)
def add_archived_books(book_id):
archived_book = (
ub.session.query(ub.ArchivedBook)
.filter(ub.ArchivedBook.book_id == book_id)
.first()
)
def change_archived_books(book_id, state=None, message=None):
archived_book = ub.session.query(ub.ArchivedBook).filter(and_(ub.ArchivedBook.user_id == int(current_user.id),
ub.ArchivedBook.book_id == book_id)).first()
if not archived_book:
archived_book = ub.ArchivedBook(user_id=current_user.id, book_id=book_id)
archived_book.is_archived = True
archived_book.last_modified = datetime.datetime.utcnow()
archived_book.is_archived = state if state else not archived_book.is_archived
archived_book.last_modified = datetime.datetime.utcnow() # toDo. Check utc timestamp
ub.session.merge(archived_book)
ub.session_commit()
ub.session_commit(message)
return archived_book.is_archived
# select all books which are synced by the current user and do not belong to a synced shelf and them to archive
# select all shelfs from current user which are synced and do not belong to the "only sync" shelfs
def update_on_sync_shelfs(content_id):
# select all books which are synced by the current user and do not belong to a synced shelf and set them to archive
# select all shelves from current user which are synced and do not belong to the "only sync" shelves
def update_on_sync_shelfs(user_id):
books_to_archive = (ub.session.query(ub.KoboSyncedBooks)
.join(ub.BookShelf, ub.KoboSyncedBooks.book_id == ub.BookShelf.book_id, isouter=True)
.join(ub.Shelf, ub.Shelf.user_id == content_id, isouter=True)
.join(ub.Shelf, ub.Shelf.user_id == user_id, isouter=True)
.filter(or_(ub.Shelf.kobo_sync == 0, ub.Shelf.kobo_sync == None))
.filter(ub.KoboSyncedBooks.user_id == content_id).all())
.filter(ub.KoboSyncedBooks.user_id == user_id).all())
for b in books_to_archive:
add_archived_books(b.book_id)
ub.session.query(ub.KoboSyncedBooks).filter(ub.KoboSyncedBooks.book_id == b.book_id).filter(ub.KoboSyncedBooks.user_id == content_id).delete()
change_archived_books(b.book_id, True)
ub.session.query(ub.KoboSyncedBooks) \
.filter(ub.KoboSyncedBooks.book_id == b.book_id) \
.filter(ub.KoboSyncedBooks.user_id == user_id).delete()
ub.session_commit()
shelfs_to_archive = ub.session.query(ub.Shelf).filter(ub.Shelf.user_id == content_id).filter(
# Search all shelf which are currently not synced
shelves_to_archive = ub.session.query(ub.Shelf).filter(ub.Shelf.user_id == user_id).filter(
ub.Shelf.kobo_sync == 0).all()
for a in shelfs_to_archive:
ub.session.add(ub.ShelfArchive(uuid=a.uuid, user_id=content_id))
for a in shelves_to_archive:
ub.session.add(ub.ShelfArchive(uuid=a.uuid, user_id=user_id))
ub.session_commit()

31
cps/logger.py
View File

@ -42,23 +42,17 @@ logging.addLevelName(logging.CRITICAL, "CRIT")
class _Logger(logging.Logger):
def debug_or_exception(self, message, *args, **kwargs):
def error_or_exception(self, message, stacklevel=2, *args, **kwargs):
if sys.version_info > (3, 7):
if is_debug_enabled():
self.exception(message, stacklevel=2, *args, **kwargs)
self.exception(message, stacklevel=stacklevel, *args, **kwargs)
else:
self.error(message, stacklevel=stacklevel, *args, **kwargs)
else:
self.error(message, stacklevel=2, *args, **kwargs)
elif sys.version_info > (3, 0):
if is_debug_enabled():
self.exception(message, stack_info=True, *args, **kwargs)
else:
self.error(message, *args, **kwargs)
else:
if is_debug_enabled():
self.exception(message, *args, **kwargs)
else:
self.error(message, *args, **kwargs)
def debug_no_auth(self, message, *args, **kwargs):
message = message.strip("\r\n")
@ -71,6 +65,7 @@ class _Logger(logging.Logger):
def get(name=None):
return logging.getLogger(name)
def create():
parent_frame = inspect.stack(0)[1]
if hasattr(parent_frame, 'frame'):
@ -80,9 +75,11 @@ def create():
parent_module = inspect.getmodule(parent_frame)
return get(parent_module.__name__)
def is_debug_enabled():
return logging.root.level <= logging.DEBUG
def is_info_enabled(logger):
return logging.getLogger(logger).level <= logging.INFO
@ -119,10 +116,10 @@ def get_accesslogfile(log_file):
def setup(log_file, log_level=None):
'''
"""
Configure the logging output.
May be called multiple times.
'''
"""
log_level = log_level or DEFAULT_LOG_LEVEL
logging.setLoggerClass(_Logger)
logging.getLogger(__package__).setLevel(log_level)
@ -132,7 +129,7 @@ def setup(log_file, log_level=None):
# avoid spamming the log with debug messages from libraries
r.setLevel(log_level)
# Otherwise name get's destroyed on windows
# Otherwise, name gets destroyed on Windows
if log_file != LOG_TO_STDERR and log_file != LOG_TO_STDOUT:
log_file = _absolute_log_file(log_file, DEFAULT_LOG_FILE)
@ -164,13 +161,14 @@ def setup(log_file, log_level=None):
r.removeHandler(h)
h.close()
r.addHandler(file_handler)
logging.captureWarnings(True)
return "" if log_file == DEFAULT_LOG_FILE else log_file
def create_access_log(log_file, log_name, formatter):
'''
"""
One-time configuration for the web server's access log.
'''
"""
log_file = _absolute_log_file(log_file, DEFAULT_ACCESS_LOG)
logging.debug("access log: %s", log_file)
@ -187,8 +185,7 @@ def create_access_log(log_file, log_name, formatter):
file_handler.setFormatter(formatter)
access_log.addHandler(file_handler)
return access_log, \
"" if _absolute_log_file(log_file, DEFAULT_ACCESS_LOG) == DEFAULT_ACCESS_LOG else log_file
return access_log, "" if _absolute_log_file(log_file, DEFAULT_ACCESS_LOG) == DEFAULT_ACCESS_LOG else log_file
# Enable logging of smtp lib debug output

73
cps/main.py Normal file
View File

@ -0,0 +1,73 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2012-2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
from . import create_app
from .jinjia import jinjia
from .remotelogin import remotelogin
def main():
app = create_app()
from .web import web
from .opds import opds
from .admin import admi
from .gdrive import gdrive
from .editbooks import editbook
from .about import about
from .search import search
from .search_metadata import meta
from .shelf import shelf
from .tasks_status import tasks
from .error_handler import init_errorhandler
try:
from .kobo import kobo, get_kobo_activated
from .kobo_auth import kobo_auth
kobo_available = get_kobo_activated()
except (ImportError, AttributeError): # Catch also error for not installed flask-WTF (missing csrf decorator)
kobo_available = False
try:
from .oauth_bb import oauth
oauth_available = True
except ImportError:
oauth_available = False
from . import web_server
init_errorhandler()
app.register_blueprint(search)
app.register_blueprint(tasks)
app.register_blueprint(web)
app.register_blueprint(opds)
app.register_blueprint(jinjia)
app.register_blueprint(about)
app.register_blueprint(shelf)
app.register_blueprint(admi)
app.register_blueprint(remotelogin)
app.register_blueprint(meta)
app.register_blueprint(gdrive)
app.register_blueprint(editbook)
if kobo_available:
app.register_blueprint(kobo)
app.register_blueprint(kobo_auth)
if oauth_available:
app.register_blueprint(oauth)
success = web_server.start()
sys.exit(0 if success else 1)

141
cps/metadata_provider/amazon.py Normal file
View File

@ -0,0 +1,141 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 quarz12
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import concurrent.futures
import requests
from bs4 import BeautifulSoup as BS # requirement
from typing import List, Optional
try:
import cchardet #optional for better speed
except ImportError:
pass
from cps import logger
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
import cps.logger as logger
#from time import time
from operator import itemgetter
log = logger.create()
log = logger.create()
class Amazon(Metadata):
__name__ = "Amazon"
__id__ = "amazon"
headers = {'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'sec-gpc': '1',
'sec-fetch-site': 'none',
'sec-fetch-mode': 'navigate',
'sec-fetch-user': '?1',
'sec-fetch-dest': 'document',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9'}
session = requests.Session()
session.headers=headers
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
#timer=time()
def inner(link, index) -> [dict, int]:
with self.session as session:
try:
r = session.get(f"https://www.amazon.com/{link}")
r.raise_for_status()
except Exception as ex:
log.warning(ex)
return None
long_soup = BS(r.text, "lxml") #~4sec :/
soup2 = long_soup.find("div", attrs={"cel_widget_id": "dpx-books-ppd_csm_instrumentation_wrapper"})
if soup2 is None:
return None
try:
match = MetaRecord(
title = "",
authors = "",
source=MetaSourceInfo(
id=self.__id__,
description="Amazon Books",
link="https://amazon.com/"
),
url = f"https://www.amazon.com{link}",
#the more searches the slower, these are too hard to find in reasonable time or might not even exist
publisher= "", # very unreliable
publishedDate= "", # very unreliable
id = None, # ?
tags = [] # dont exist on amazon
)
try:
match.description = "\n".join(
soup2.find("div", attrs={"data-feature-name": "bookDescription"}).stripped_strings)\
.replace("\xa0"," ")[:-9].strip().strip("\n")
except (AttributeError, TypeError):
return None # if there is no description it is not a book and therefore should be ignored
try:
match.title = soup2.find("span", attrs={"id": "productTitle"}).text
except (AttributeError, TypeError):
match.title = ""
try:
match.authors = [next(
filter(lambda i: i != " " and i != "\n" and not i.startswith("{"),
x.findAll(text=True))).strip()
for x in soup2.findAll("span", attrs={"class": "author"})]
except (AttributeError, TypeError, StopIteration):
match.authors = ""
try:
match.rating = int(
soup2.find("span", class_="a-icon-alt").text.split(" ")[0].split(".")[
0]) # first number in string
except (AttributeError, ValueError):
match.rating = 0
try:
match.cover = soup2.find("img", attrs={"class": "a-dynamic-image frontImage"})["src"]
except (AttributeError, TypeError):
match.cover = ""
return match, index
except Exception as e:
log.error_or_exception(e)
return None
val = list()
if self.active:
try:
results = self.session.get(
f"https://www.amazon.com/s?k={query.replace(' ', '+')}&i=digital-text&sprefix={query.replace(' ', '+')}"
f"%2Cdigital-text&ref=nb_sb_noss",
headers=self.headers)
results.raise_for_status()
except requests.exceptions.HTTPError as e:
log.error_or_exception(e)
return []
except Exception as e:
log.warning(e)
return []
soup = BS(results.text, 'html.parser')
links_list = [next(filter(lambda i: "digital-text" in i["href"], x.findAll("a")))["href"] for x in
soup.findAll("div", attrs={"data-component-type": "s-search-result"})]
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
fut = {executor.submit(inner, link, index) for index, link in enumerate(links_list[:5])}
val = list(map(lambda x : x.result() ,concurrent.futures.as_completed(fut)))
result = list(filter(lambda x: x, val))
return [x[0] for x in sorted(result, key=itemgetter(1))] #sort by amazons listing order for best relevance

95
cps/metadata_provider/comicvine.py
View File

@ -17,49 +17,76 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# ComicVine api document: https://comicvine.gamespot.com/api/documentation
from typing import Dict, List, Optional
from urllib.parse import quote
import requests
from cps.services.Metadata import Metadata
from cps import logger
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
class ComicVine(Metadata):
__name__ = "ComicVine"
__id__ = "comicvine"
DESCRIPTION = "ComicVine Books"
META_URL = "https://comicvine.gamespot.com/"
API_KEY = "57558043c53943d5d1e96a9ad425b0eb85532ee6"
BASE_URL = (
f"https://comicvine.gamespot.com/api/search?api_key={API_KEY}"
f"&resources=issue&query="
)
QUERY_PARAMS = "&sort=name:desc&format=json"
HEADERS = {"User-Agent": "Not Evil Browser"}
def search(self, query, __):
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
val = list()
apikey = "57558043c53943d5d1e96a9ad425b0eb85532ee6"
if self.active:
headers = {
'User-Agent': 'Not Evil Browser'
}
result = requests.get("https://comicvine.gamespot.com/api/search?api_key="
+ apikey + "&resources=issue&query=" + query + "&sort=name:desc&format=json", headers=headers)
for r in result.json()['results']:
seriesTitle = r['volume'].get('name', "")
if r.get('store_date'):
dateFomers = r.get('store_date')
else:
dateFomers = r.get('date_added')
v = dict()
v['id'] = r['id']
v['title'] = seriesTitle + " #" + r.get('issue_number', "0") + " - " + ( r.get('name', "") or "")
v['authors'] = r.get('authors', [])
v['description'] = r.get('description', "")
v['publisher'] = ""
v['publishedDate'] = dateFomers
v['tags'] = ["Comics", seriesTitle]
v['rating'] = 0
v['series'] = seriesTitle
v['cover'] = r['image'].get('original_url')
v['source'] = {
"id": self.__id__,
"description": "ComicVine Books",
"link": "https://comicvine.gamespot.com/"
}
v['url'] = r.get('site_detail_url', "")
val.append(v)
title_tokens = list(self.get_title_tokens(query, strip_joiners=False))
if title_tokens:
tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = "%20".join(tokens)
try:
result = requests.get(
f"{ComicVine.BASE_URL}{query}{ComicVine.QUERY_PARAMS}",
headers=ComicVine.HEADERS,
)
result.raise_for_status()
except Exception as e:
log.warning(e)
return None
for result in result.json()["results"]:
match = self._parse_search_result(
result=result, generic_cover=generic_cover, locale=locale
)
val.append(match)
return val
def _parse_search_result(
self, result: Dict, generic_cover: str, locale: str
) -> MetaRecord:
series = result["volume"].get("name", "")
series_index = result.get("issue_number", 0)
issue_name = result.get("name", "")
match = MetaRecord(
id=result["id"],
title=f"{series}#{series_index} - {issue_name}",
authors=result.get("authors", []),
url=result.get("site_detail_url", ""),
source=MetaSourceInfo(
id=self.__id__,
description=ComicVine.DESCRIPTION,
link=ComicVine.META_URL,
),
series=series,
)
match.cover = result["image"].get("original_url", generic_cover)
match.description = result.get("description", "")
match.publishedDate = result.get("store_date", result.get("date_added"))
match.series_index = series_index
match.tags = ["Comics", series]
match.identifiers = {"comicvine": match.id}
return match

258
cps/metadata_provider/douban.py Normal file
View File

@ -0,0 +1,258 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 xlivevil
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import re
from concurrent import futures
from typing import List, Optional
import requests
from html2text import HTML2Text
from lxml import etree
from cps import logger
from cps.services.Metadata import Metadata, MetaRecord, MetaSourceInfo
log = logger.create()
def html2text(html: str) -> str:
h2t = HTML2Text()
h2t.body_width = 0
h2t.single_line_break = True
h2t.emphasis_mark = "*"
return h2t.handle(html)
class Douban(Metadata):
__name__ = "豆瓣"
__id__ = "douban"
DESCRIPTION = "豆瓣"
META_URL = "https://book.douban.com/"
SEARCH_JSON_URL = "https://www.douban.com/j/search"
SEARCH_URL = "https://www.douban.com/search"
ID_PATTERN = re.compile(r"sid: (?P<id>\d+),")
AUTHORS_PATTERN = re.compile(r"作者|译者")
PUBLISHER_PATTERN = re.compile(r"出版社")
SUBTITLE_PATTERN = re.compile(r"副标题")
PUBLISHED_DATE_PATTERN = re.compile(r"出版年")
SERIES_PATTERN = re.compile(r"丛书")
IDENTIFIERS_PATTERN = re.compile(r"ISBN|统一书号")
CRITERIA_PATTERN = re.compile("criteria = '(.+)'")
TITTLE_XPATH = "//span[@property='v:itemreviewed']"
COVER_XPATH = "//a[@class='nbg']"
INFO_XPATH = "//*[@id='info']//span[@class='pl']"
TAGS_XPATH = "//a[contains(@class, 'tag')]"
DESCRIPTION_XPATH = "//div[@id='link-report']//div[@class='intro']"
RATING_XPATH = "//div[@class='rating_self clearfix']/strong"
session = requests.Session()
session.headers = {
'user-agent':
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36 Edg/98.0.1108.56',
}
def search(self,
query: str,
generic_cover: str = "",
locale: str = "en") -> List[MetaRecord]:
val = []
if self.active:
log.debug(f"start searching {query} on douban")
if title_tokens := list(
self.get_title_tokens(query, strip_joiners=False)):
query = "+".join(title_tokens)
book_id_list = self._get_book_id_list_from_html(query)
if not book_id_list:
log.debug("No search results in Douban")
return []
with futures.ThreadPoolExecutor(
max_workers=5, thread_name_prefix='douban') as executor:
fut = [
executor.submit(self._parse_single_book, book_id,
generic_cover) for book_id in book_id_list
]
val = [
future.result() for future in futures.as_completed(fut)
if future.result()
]
return val
def _get_book_id_list_from_html(self, query: str) -> List[str]:
try:
r = self.session.get(self.SEARCH_URL,
params={
"cat": 1001,
"q": query
})
r.raise_for_status()
except Exception as e:
log.warning(e)
return []
html = etree.HTML(r.content.decode("utf8"))
result_list = html.xpath(self.COVER_XPATH)
return [
self.ID_PATTERN.search(item.get("onclick")).group("id")
for item in result_list[:10]
if self.ID_PATTERN.search(item.get("onclick"))
]
def _get_book_id_list_from_json(self, query: str) -> List[str]:
try:
r = self.session.get(self.SEARCH_JSON_URL,
params={
"cat": 1001,
"q": query
})
r.raise_for_status()
except Exception as e:
log.warning(e)
return []
results = r.json()
if results["total"] == 0:
return []
return [
self.ID_PATTERN.search(item).group("id")
for item in results["items"][:10] if self.ID_PATTERN.search(item)
]
def _parse_single_book(self,
id: str,
generic_cover: str = "") -> Optional[MetaRecord]:
url = f"https://book.douban.com/subject/{id}/"
log.debug(f"start parsing {url}")
try:
r = self.session.get(url)
r.raise_for_status()
except Exception as e:
log.warning(e)
return None
match = MetaRecord(
id=id,
title="",
authors=[],
url=url,
source=MetaSourceInfo(
id=self.__id__,
description=self.DESCRIPTION,
link=self.META_URL,
),
)
html = etree.HTML(r.content.decode("utf8"))
match.title = html.xpath(self.TITTLE_XPATH)[0].text
match.cover = html.xpath(
self.COVER_XPATH)[0].attrib["href"] or generic_cover
try:
rating_num = float(html.xpath(self.RATING_XPATH)[0].text.strip())
except Exception:
rating_num = 0
match.rating = int(-1 * rating_num // 2 * -1) if rating_num else 0
tag_elements = html.xpath(self.TAGS_XPATH)
if len(tag_elements):
match.tags = [tag_element.text for tag_element in tag_elements]
else:
match.tags = self._get_tags(html.text)
description_element = html.xpath(self.DESCRIPTION_XPATH)
if len(description_element):
match.description = html2text(
etree.tostring(description_element[-1]).decode("utf8"))
info = html.xpath(self.INFO_XPATH)
for element in info:
text = element.text
if self.AUTHORS_PATTERN.search(text):
next_element = element.getnext()
while next_element is not None and next_element.tag != "br":
match.authors.append(next_element.text)
next_element = next_element.getnext()
elif self.PUBLISHER_PATTERN.search(text):
if publisher := element.tail.strip():
match.publisher = publisher
else:
match.publisher = element.getnext().text
elif self.SUBTITLE_PATTERN.search(text):
match.title = f'{match.title}:{element.tail.strip()}'
elif self.PUBLISHED_DATE_PATTERN.search(text):
match.publishedDate = self._clean_date(element.tail.strip())
elif self.SERIES_PATTERN.search(text):
match.series = element.getnext().text
elif i_type := self.IDENTIFIERS_PATTERN.search(text):
match.identifiers[i_type.group()] = element.tail.strip()
return match
def _clean_date(self, date: str) -> str:
"""
Clean up the date string to be in the format YYYY-MM-DD
Examples of possible patterns:
'2014-7-16', '1988年4月', '1995-04', '2021-8', '2020-12-1', '1996年',
'1972', '2004/11/01', '1959年3月北京第1版第1印'
"""
year = date[:4]
moon = "01"
day = "01"
if len(date) > 5:
digit = []
ls = []
for i in range(5, len(date)):
if date[i].isdigit():
digit.append(date[i])
elif digit:
ls.append("".join(digit) if len(digit) ==
2 else f"0{digit[0]}")
digit = []
if digit:
ls.append("".join(digit) if len(digit) ==
2 else f"0{digit[0]}")
moon = ls[0]
if len(ls) > 1:
day = ls[1]
return f"{year}-{moon}-{day}"
def _get_tags(self, text: str) -> List[str]:
tags = []
if criteria := self.CRITERIA_PATTERN.search(text):
tags.extend(
item.replace('7:', '') for item in criteria.group().split('|')
if item.startswith('7:'))
return tags

124
cps/metadata_provider/google.py
View File

@ -17,39 +17,113 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Google Books api document: https://developers.google.com/books/docs/v1/using
from typing import Dict, List, Optional
from urllib.parse import quote
from datetime import datetime
import requests
from cps.services.Metadata import Metadata
from cps import logger
from cps.isoLanguages import get_lang3, get_language_name
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
class Google(Metadata):
__name__ = "Google"
__id__ = "google"
DESCRIPTION = "Google Books"
META_URL = "https://books.google.com/"
BOOK_URL = "https://books.google.com/books?id="
SEARCH_URL = "https://www.googleapis.com/books/v1/volumes?q="
ISBN_TYPE = "ISBN_13"
def search(self, query, __):
if self.active:
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
val = list()
result = requests.get("https://www.googleapis.com/books/v1/volumes?q="+query.replace(" ","+"))
for r in result.json()['items']:
v = dict()
v['id'] = r['id']
v['title'] = r['volumeInfo']['title']
v['authors'] = r['volumeInfo'].get('authors', [])
v['description'] = r['volumeInfo'].get('description', "")
v['publisher'] = r['volumeInfo'].get('publisher', "")
v['publishedDate'] = r['volumeInfo'].get('publishedDate', "")
v['tags'] = r['volumeInfo'].get('categories', [])
v['rating'] = r['volumeInfo'].get('averageRating', 0)
if r['volumeInfo'].get('imageLinks'):
v['cover'] = r['volumeInfo']['imageLinks']['thumbnail'].replace("http://", "https://")
else:
v['cover'] = "/../../../static/generic_cover.jpg"
v['source'] = {
"id": self.__id__,
"description": "Google Books",
"link": "https://books.google.com/"}
v['url'] = "https://books.google.com/books?id=" + r['id']
val.append(v)
if self.active:
title_tokens = list(self.get_title_tokens(query, strip_joiners=False))
if title_tokens:
tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = "+".join(tokens)
try:
results = requests.get(Google.SEARCH_URL + query)
results.raise_for_status()
except Exception as e:
log.warning(e)
return None
for result in results.json().get("items", []):
val.append(
self._parse_search_result(
result=result, generic_cover=generic_cover, locale=locale
)
)
return val
def _parse_search_result(
self, result: Dict, generic_cover: str, locale: str
) -> MetaRecord:
match = MetaRecord(
id=result["id"],
title=result["volumeInfo"]["title"],
authors=result["volumeInfo"].get("authors", []),
url=Google.BOOK_URL + result["id"],
source=MetaSourceInfo(
id=self.__id__,
description=Google.DESCRIPTION,
link=Google.META_URL,
),
)
match.cover = self._parse_cover(result=result, generic_cover=generic_cover)
match.description = result["volumeInfo"].get("description", "")
match.languages = self._parse_languages(result=result, locale=locale)
match.publisher = result["volumeInfo"].get("publisher", "")
try:
datetime.strptime(result["volumeInfo"].get("publishedDate", ""), "%Y-%m-%d")
match.publishedDate = result["volumeInfo"].get("publishedDate", "")
except ValueError:
match.publishedDate = ""
match.rating = result["volumeInfo"].get("averageRating", 0)
match.series, match.series_index = "", 1
match.tags = result["volumeInfo"].get("categories", [])
match.identifiers = {"google": match.id}
match = self._parse_isbn(result=result, match=match)
return match
@staticmethod
def _parse_isbn(result: Dict, match: MetaRecord) -> MetaRecord:
identifiers = result["volumeInfo"].get("industryIdentifiers", [])
for identifier in identifiers:
if identifier.get("type") == Google.ISBN_TYPE:
match.identifiers["isbn"] = identifier.get("identifier")
break
return match
@staticmethod
def _parse_cover(result: Dict, generic_cover: str) -> str:
if result["volumeInfo"].get("imageLinks"):
cover_url = result["volumeInfo"]["imageLinks"]["thumbnail"]
# strip curl in cover
cover_url = cover_url.replace("&edge=curl", "")
# request 800x900 cover image (higher resolution)
cover_url += "&fife=w800-h900"
return cover_url.replace("http://", "https://")
return generic_cover
@staticmethod
def _parse_languages(result: Dict, locale: str) -> List[str]:
language_iso2 = result["volumeInfo"].get("language", "")
languages = (
[get_language_name(locale, get_lang3(language_iso2))]
if language_iso2
else []
)
return languages

350
cps/metadata_provider/lubimyczytac.py Normal file
View File

@ -0,0 +1,350 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2021 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import datetime
import json
import re
from multiprocessing.pool import ThreadPool
from typing import List, Optional, Tuple, Union
from urllib.parse import quote
import requests
from dateutil import parser
from html2text import HTML2Text
from lxml.html import HtmlElement, fromstring, tostring
from markdown2 import Markdown
from cps import logger
from cps.isoLanguages import get_language_name
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
SYMBOLS_TO_TRANSLATE = (
"öÖüÜóÓőŐúÚéÉáÁűŰíÍąĄćĆęĘłŁńŃóÓśŚźŹżŻ",
"oOuUoOoOuUeEaAuUiIaAcCeElLnNoOsSzZzZ",
)
SYMBOL_TRANSLATION_MAP = dict(
[(ord(a), ord(b)) for (a, b) in zip(*SYMBOLS_TO_TRANSLATE)]
)
def get_int_or_float(value: str) -> Union[int, float]:
number_as_float = float(value)
number_as_int = int(number_as_float)
return number_as_int if number_as_float == number_as_int else number_as_float
def strip_accents(s: Optional[str]) -> Optional[str]:
return s.translate(SYMBOL_TRANSLATION_MAP) if s is not None else s
def sanitize_comments_html(html: str) -> str:
text = html2text(html)
md = Markdown()
html = md.convert(text)
return html
def html2text(html: str) -> str:
# replace <u> tags with <span> as <u> becomes emphasis in html2text
if isinstance(html, bytes):
html = html.decode("utf-8")
html = re.sub(
r"<\s*(?P<solidus>/?)\s*[uU]\b(?P<rest>[^>]*)>",
r"<\g<solidus>span\g<rest>>",
html,
)
h2t = HTML2Text()
h2t.body_width = 0
h2t.single_line_break = True
h2t.emphasis_mark = "*"
return h2t.handle(html)
class LubimyCzytac(Metadata):
__name__ = "LubimyCzytac.pl"
__id__ = "lubimyczytac"
BASE_URL = "https://lubimyczytac.pl"
BOOK_SEARCH_RESULT_XPATH = (
"*//div[@class='listSearch']//div[@class='authorAllBooks__single']"
)
SINGLE_BOOK_RESULT_XPATH = ".//div[contains(@class,'authorAllBooks__singleText')]"
TITLE_PATH = "/div/a[contains(@class,'authorAllBooks__singleTextTitle')]"
TITLE_TEXT_PATH = f"{TITLE_PATH}//text()"
URL_PATH = f"{TITLE_PATH}/@href"
AUTHORS_PATH = "/div/a[contains(@href,'autor')]//text()"
SIBLINGS = "/following-sibling::dd"
CONTAINER = "//section[@class='container book']"
PUBLISHER = f"{CONTAINER}//dt[contains(text(),'Wydawnictwo:')]{SIBLINGS}/a/text()"
LANGUAGES = f"{CONTAINER}//dt[contains(text(),'Język:')]{SIBLINGS}/text()"
DESCRIPTION = f"{CONTAINER}//div[@class='collapse-content']"
SERIES = f"{CONTAINER}//span/a[contains(@href,'/cykl/')]/text()"
DETAILS = "//div[@id='book-details']"
PUBLISH_DATE = "//dt[contains(@title,'Data pierwszego wydania"
FIRST_PUBLISH_DATE = f"{DETAILS}{PUBLISH_DATE} oryginalnego')]{SIBLINGS}[1]/text()"
FIRST_PUBLISH_DATE_PL = f"{DETAILS}{PUBLISH_DATE} polskiego')]{SIBLINGS}[1]/text()"
TAGS = "//nav[@aria-label='breadcrumb']//a[contains(@href,'/ksiazki/k/')]/text()"
RATING = "//meta[@property='books:rating:value']/@content"
COVER = "//meta[@property='og:image']/@content"
ISBN = "//meta[@property='books:isbn']/@content"
META_TITLE = "//meta[@property='og:description']/@content"
SUMMARY = "//script[@type='application/ld+json']//text()"
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
if self.active:
try:
result = requests.get(self._prepare_query(title=query))
result.raise_for_status()
except Exception as e:
log.warning(e)
return None
root = fromstring(result.text)
lc_parser = LubimyCzytacParser(root=root, metadata=self)
matches = lc_parser.parse_search_results()
if matches:
with ThreadPool(processes=10) as pool:
final_matches = pool.starmap(
lc_parser.parse_single_book,
[(match, generic_cover, locale) for match in matches],
)
return final_matches
return matches
def _prepare_query(self, title: str) -> str:
query = ""
characters_to_remove = "\?()\/"
pattern = "[" + characters_to_remove + "]"
title = re.sub(pattern, "", title)
title = title.replace("_", " ")
if '"' in title or ",," in title:
title = title.split('"')[0].split(",,")[0]
if "/" in title:
title_tokens = [
token for token in title.lower().split(" ") if len(token) > 1
]
else:
title_tokens = list(self.get_title_tokens(title, strip_joiners=False))
if title_tokens:
tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = query + "%20".join(tokens)
if not query:
return ""
return f"{LubimyCzytac.BASE_URL}/szukaj/ksiazki?phrase={query}"
class LubimyCzytacParser:
PAGES_TEMPLATE = "<p id='strony'>Książka ma {0} stron(y).</p>"
PUBLISH_DATE_TEMPLATE = "<p id='pierwsze_wydanie'>Data pierwszego wydania: {0}</p>"
PUBLISH_DATE_PL_TEMPLATE = (
"<p id='pierwsze_wydanie'>Data pierwszego wydania w Polsce: {0}</p>"
)
def __init__(self, root: HtmlElement, metadata: Metadata) -> None:
self.root = root
self.metadata = metadata
def parse_search_results(self) -> List[MetaRecord]:
matches = []
results = self.root.xpath(LubimyCzytac.BOOK_SEARCH_RESULT_XPATH)
for result in results:
title = self._parse_xpath_node(
root=result,
xpath=f"{LubimyCzytac.SINGLE_BOOK_RESULT_XPATH}"
f"{LubimyCzytac.TITLE_TEXT_PATH}",
)
book_url = self._parse_xpath_node(
root=result,
xpath=f"{LubimyCzytac.SINGLE_BOOK_RESULT_XPATH}"
f"{LubimyCzytac.URL_PATH}",
)
authors = self._parse_xpath_node(
root=result,
xpath=f"{LubimyCzytac.SINGLE_BOOK_RESULT_XPATH}"
f"{LubimyCzytac.AUTHORS_PATH}",
take_first=False,
)
if not all([title, book_url, authors]):
continue
matches.append(
MetaRecord(
id=book_url.replace(f"/ksiazka/", "").split("/")[0],
title=title,
authors=[strip_accents(author) for author in authors],
url=LubimyCzytac.BASE_URL + book_url,
source=MetaSourceInfo(
id=self.metadata.__id__,
description=self.metadata.__name__,
link=LubimyCzytac.BASE_URL,
),
)
)
return matches
def parse_single_book(
self, match: MetaRecord, generic_cover: str, locale: str
) -> MetaRecord:
try:
response = requests.get(match.url)
response.raise_for_status()
except Exception as e:
log.warning(e)
return None
self.root = fromstring(response.text)
match.cover = self._parse_cover(generic_cover=generic_cover)
match.description = self._parse_description()
match.languages = self._parse_languages(locale=locale)
match.publisher = self._parse_publisher()
match.publishedDate = self._parse_from_summary(attribute_name="datePublished")
match.rating = self._parse_rating()
match.series, match.series_index = self._parse_series()
match.tags = self._parse_tags()
match.identifiers = {
"isbn": self._parse_isbn(),
"lubimyczytac": match.id,
}
return match
def _parse_xpath_node(
self,
xpath: str,
root: HtmlElement = None,
take_first: bool = True,
strip_element: bool = True,
) -> Optional[Union[str, List[str]]]:
root = root if root is not None else self.root
node = root.xpath(xpath)
if not node:
return None
return (
(node[0].strip() if strip_element else node[0])
if take_first
else [x.strip() for x in node]
)
def _parse_cover(self, generic_cover) -> Optional[str]:
return (
self._parse_xpath_node(xpath=LubimyCzytac.COVER, take_first=True)
or generic_cover
)
def _parse_publisher(self) -> Optional[str]:
return self._parse_xpath_node(xpath=LubimyCzytac.PUBLISHER, take_first=True)
def _parse_languages(self, locale: str) -> List[str]:
languages = list()
lang = self._parse_xpath_node(xpath=LubimyCzytac.LANGUAGES, take_first=True)
if lang:
if "polski" in lang:
languages.append("pol")
if "angielski" in lang:
languages.append("eng")
return [get_language_name(locale, language) for language in languages]
def _parse_series(self) -> Tuple[Optional[str], Optional[Union[float, int]]]:
series_index = 0
series = self._parse_xpath_node(xpath=LubimyCzytac.SERIES, take_first=True)
if series:
if "tom " in series:
series_name, series_info = series.split(" (tom ", 1)
series_info = series_info.replace(" ", "").replace(")", "")
# Check if book is not a bundle, i.e. chapter 1-3
if "-" in series_info:
series_info = series_info.split("-", 1)[0]
if series_info.replace(".", "").isdigit() is True:
series_index = get_int_or_float(series_info)
return series_name, series_index
return None, None
def _parse_tags(self) -> List[str]:
tags = self._parse_xpath_node(xpath=LubimyCzytac.TAGS, take_first=False)
return [
strip_accents(w.replace(", itd.", " itd."))
for w in tags
if isinstance(w, str)
]
def _parse_from_summary(self, attribute_name: str) -> Optional[str]:
value = None
summary_text = self._parse_xpath_node(xpath=LubimyCzytac.SUMMARY)
if summary_text:
data = json.loads(summary_text)
value = data.get(attribute_name)
return value.strip() if value is not None else value
def _parse_rating(self) -> Optional[str]:
rating = self._parse_xpath_node(xpath=LubimyCzytac.RATING)
return round(float(rating.replace(",", ".")) / 2) if rating else rating
def _parse_date(self, xpath="first_publish") -> Optional[datetime.datetime]:
options = {
"first_publish": LubimyCzytac.FIRST_PUBLISH_DATE,
"first_publish_pl": LubimyCzytac.FIRST_PUBLISH_DATE_PL,
}
date = self._parse_xpath_node(xpath=options.get(xpath))
return parser.parse(date) if date else None
def _parse_isbn(self) -> Optional[str]:
return self._parse_xpath_node(xpath=LubimyCzytac.ISBN)
def _parse_description(self) -> str:
description = ""
description_node = self._parse_xpath_node(
xpath=LubimyCzytac.DESCRIPTION, strip_element=False
)
if description_node is not None:
for source in self.root.xpath('//p[@class="source"]'):
source.getparent().remove(source)
description = tostring(description_node, method="html")
description = sanitize_comments_html(description)
else:
description_node = self._parse_xpath_node(xpath=LubimyCzytac.META_TITLE)
if description_node is not None:
description = description_node
description = sanitize_comments_html(description)
description = self._add_extra_info_to_description(description=description)
return description
def _add_extra_info_to_description(self, description: str) -> str:
pages = self._parse_from_summary(attribute_name="numberOfPages")
if pages:
description += LubimyCzytacParser.PAGES_TEMPLATE.format(pages)
first_publish_date = self._parse_date()
if first_publish_date:
description += LubimyCzytacParser.PUBLISH_DATE_TEMPLATE.format(
first_publish_date.strftime("%d.%m.%Y")
)
first_publish_date_pl = self._parse_date(xpath="first_publish_pl")
if first_publish_date_pl:
description += LubimyCzytacParser.PUBLISH_DATE_PL_TEMPLATE.format(
first_publish_date_pl.strftime("%d.%m.%Y")
)
return description

84
cps/metadata_provider/scholar.py
View File

@ -15,47 +15,69 @@
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import itertools
from typing import Dict, List, Optional
from urllib.parse import quote, unquote
from scholarly import scholarly
try:
from fake_useragent.errors import FakeUserAgentError
except (ImportError):
FakeUserAgentError = BaseException
try:
from scholarly import scholarly
except FakeUserAgentError:
raise ImportError("No module named 'scholarly'")
from cps.services.Metadata import Metadata
from cps import logger
from cps.services.Metadata import MetaRecord, MetaSourceInfo, Metadata
log = logger.create()
class scholar(Metadata):
__name__ = "Google Scholar"
__id__ = "googlescholar"
META_URL = "https://scholar.google.com/"
def search(self, query, generic_cover=""):
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
val = list()
if self.active:
scholar_gen = scholarly.search_pubs(' '.join(query.split('+')))
i = 0
for publication in scholar_gen:
v = dict()
v['id'] = "1234" # publication['bib'].get('title')
v['title'] = publication['bib'].get('title')
v['authors'] = publication['bib'].get('author', [])
v['description'] = publication['bib'].get('abstract', "")
v['publisher'] = publication['bib'].get('venue', "")
if publication['bib'].get('pub_year'):
v['publishedDate'] = publication['bib'].get('pub_year')+"-01-01"
else:
v['publishedDate'] = ""
v['tags'] = ""
v['ratings'] = 0
v['series'] = ""
v['cover'] = generic_cover
v['url'] = publication.get('pub_url') or publication.get('eprint_url') or "",
v['source'] = {
"id": self.__id__,
"description": "Google Scholar",
"link": "https://scholar.google.com/"
}
val.append(v)
i += 1
if (i >= 10):
break
title_tokens = list(self.get_title_tokens(query, strip_joiners=False))
if title_tokens:
tokens = [quote(t.encode("utf-8")) for t in title_tokens]
query = " ".join(tokens)
try:
scholarly.set_timeout(20)
scholarly.set_retries(2)
scholar_gen = itertools.islice(scholarly.search_pubs(query), 10)
except Exception as e:
log.warning(e)
return list()
for result in scholar_gen:
match = self._parse_search_result(
result=result, generic_cover="", locale=locale
)
val.append(match)
return val
def _parse_search_result(
self, result: Dict, generic_cover: str, locale: str
) -> MetaRecord:
match = MetaRecord(
id=result.get("pub_url", result.get("eprint_url", "")),
title=result["bib"].get("title"),
authors=result["bib"].get("author", []),
url=result.get("pub_url", result.get("eprint_url", "")),
source=MetaSourceInfo(
id=self.__id__, description=self.__name__, link=scholar.META_URL
),
)
match.cover = result.get("image", {}).get("original_url", generic_cover)
match.description = unquote(result["bib"].get("abstract", ""))
match.publisher = result["bib"].get("venue", "")
match.publishedDate = result["bib"].get("pub_year") + "-01-01"
match.identifiers = {"scholar": match.id}
return match

8
cps/oauth.py
View File

@ -19,17 +19,11 @@
from flask import session
try:
from flask_dance.consumer.backend.sqla import SQLAlchemyBackend, first, _get_real_user
from sqlalchemy.orm.exc import NoResultFound
backend_resultcode = False # prevent storing values with this resultcode
except ImportError:
# fails on flask-dance >1.3, due to renaming
try:
from flask_dance.consumer.storage.sqla import SQLAlchemyStorage as SQLAlchemyBackend
from flask_dance.consumer.storage.sqla import first, _get_real_user
from sqlalchemy.orm.exc import NoResultFound
backend_resultcode = True # prevent storing values with this resultcode
except ImportError:
except ImportError:
pass

46
cps/oauth_bb.py
View File

@ -74,7 +74,7 @@ def register_user_with_oauth(user=None):
if len(all_oauth.keys()) == 0:
return
if user is None:
flash(_(u"Register with %(provider)s", provider=", ".join(list(all_oauth.values()))), category="success")
flash(_("Register with %(provider)s", provider=", ".join(list(all_oauth.values()))), category="success")
else:
for oauth_key in all_oauth.keys():
# Find this OAuth token in the database, or create it
@ -134,8 +134,8 @@ def bind_oauth_or_register(provider_id, provider_user_id, redirect_url, provider
# already bind with user, just login
if oauth_entry.user:
login_user(oauth_entry.user)
log.debug(u"You are now logged in as: '%s'", oauth_entry.user.name)
flash(_(u"you are now logged in as: '%(nickname)s'", nickname= oauth_entry.user.name),
log.debug("You are now logged in as: '%s'", oauth_entry.user.name)
flash(_("Success! You are now logged in as: %(nickname)s", nickname= oauth_entry.user.name),
category="success")
return redirect(url_for('web.index'))
else:
@ -145,21 +145,21 @@ def bind_oauth_or_register(provider_id, provider_user_id, redirect_url, provider
try:
ub.session.add(oauth_entry)
ub.session.commit()
flash(_(u"Link to %(oauth)s Succeeded", oauth=provider_name), category="success")
flash(_("Link to %(oauth)s Succeeded", oauth=provider_name), category="success")
log.info("Link to {} Succeeded".format(provider_name))
return redirect(url_for('web.profile'))
except Exception as ex:
log.debug_or_exception(ex)
log.error_or_exception(ex)
ub.session.rollback()
else:
flash(_(u"Login failed, No User Linked With OAuth Account"), category="error")
flash(_("Login failed, No User Linked With OAuth Account"), category="error")
log.info('Login failed, No User Linked With OAuth Account')
return redirect(url_for('web.login'))
# return redirect(url_for('web.login'))
# if config.config_public_reg:
# return redirect(url_for('web.register'))
# else:
# flash(_(u"Public registration is not enabled"), category="error")
# flash(_("Public registration is not enabled"), category="error")
# return redirect(url_for(redirect_url))
except (NoResultFound, AttributeError):
return redirect(url_for(redirect_url))
@ -194,15 +194,15 @@ def unlink_oauth(provider):
ub.session.delete(oauth_entry)
ub.session.commit()
logout_oauth_user()
flash(_(u"Unlink to %(oauth)s Succeeded", oauth=oauth_check[provider]), category="success")
flash(_("Unlink to %(oauth)s Succeeded", oauth=oauth_check[provider]), category="success")
log.info("Unlink to {} Succeeded".format(oauth_check[provider]))
except Exception as ex:
log.debug_or_exception(ex)
log.error_or_exception(ex)
ub.session.rollback()
flash(_(u"Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error")
flash(_("Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error")
except NoResultFound:
log.warning("oauth %s for user %d not found", provider, current_user.id)
flash(_(u"Not Linked to %(oauth)s", oauth=provider), category="error")
flash(_("Not Linked to %(oauth)s", oauth=provider), category="error")
return redirect(url_for('web.profile'))
def generate_oauth_blueprints():
@ -258,13 +258,13 @@ if ub.oauth_support:
@oauth_authorized.connect_via(oauthblueprints[0]['blueprint'])
def github_logged_in(blueprint, token):
if not token:
flash(_(u"Failed to log in with GitHub."), category="error")
flash(_("Failed to log in with GitHub."), category="error")
log.error("Failed to log in with GitHub")
return False
resp = blueprint.session.get("/user")
if not resp.ok:
flash(_(u"Failed to fetch user info from GitHub."), category="error")
flash(_("Failed to fetch user info from GitHub."), category="error")
log.error("Failed to fetch user info from GitHub")
return False
@ -276,13 +276,13 @@ if ub.oauth_support:
@oauth_authorized.connect_via(oauthblueprints[1]['blueprint'])
def google_logged_in(blueprint, token):
if not token:
flash(_(u"Failed to log in with Google."), category="error")
flash(_("Failed to log in with Google."), category="error")
log.error("Failed to log in with Google")
return False
resp = blueprint.session.get("/oauth2/v2/userinfo")
if not resp.ok:
flash(_(u"Failed to fetch user info from Google."), category="error")
flash(_("Failed to fetch user info from Google."), category="error")
log.error("Failed to fetch user info from Google")
return False
@ -295,8 +295,8 @@ if ub.oauth_support:
@oauth_error.connect_via(oauthblueprints[0]['blueprint'])
def github_error(blueprint, error, error_description=None, error_uri=None):
msg = (
u"OAuth error from {name}! "
u"error={error} description={description} uri={uri}"
"OAuth error from {name}! "
"error={error} description={description} uri={uri}"
).format(
name=blueprint.name,
error=error,
@ -308,8 +308,8 @@ if ub.oauth_support:
@oauth_error.connect_via(oauthblueprints[1]['blueprint'])
def google_error(blueprint, error, error_description=None, error_uri=None):
msg = (
u"OAuth error from {name}! "
u"error={error} description={description} uri={uri}"
"OAuth error from {name}! "
"error={error} description={description} uri={uri}"
).format(
name=blueprint.name,
error=error,
@ -329,10 +329,10 @@ def github_login():
if account_info.ok:
account_info_json = account_info.json()
return bind_oauth_or_register(oauthblueprints[0]['id'], account_info_json['id'], 'github.login', 'github')
flash(_(u"GitHub Oauth error, please retry later."), category="error")
flash(_("GitHub Oauth error, please retry later."), category="error")
log.error("GitHub Oauth error, please retry later")
except (InvalidGrantError, TokenExpiredError) as e:
flash(_(u"GitHub Oauth error: {}").format(e), category="error")
flash(_("GitHub Oauth error: {}").format(e), category="error")
log.error(e)
return redirect(url_for('web.login'))
@ -353,10 +353,10 @@ def google_login():
if resp.ok:
account_info_json = resp.json()
return bind_oauth_or_register(oauthblueprints[1]['id'], account_info_json['id'], 'google.login', 'google')
flash(_(u"Google Oauth error, please retry later."), category="error")
flash(_("Google Oauth error, please retry later."), category="error")
log.error("Google Oauth error, please retry later")
except (InvalidGrantError, TokenExpiredError) as e:
flash(_(u"Google Oauth error: {}").format(e), category="error")
flash(_("Google Oauth error: {}").format(e), category="error")
log.error(e)
return redirect(url_for('web.login'))

352
cps/opds.py
View File

@ -20,56 +20,28 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import datetime
from functools import wraps
from urllib.parse import unquote_plus
from flask import Blueprint, request, render_template, Response, g, make_response, abort
from flask import Blueprint, request, render_template, make_response, abort
from flask_login import current_user
from flask_babel import get_locale
from flask_babel import gettext as _
from sqlalchemy.sql.expression import func, text, or_, and_, true
from werkzeug.security import check_password_hash
from sqlalchemy.exc import InvalidRequestError, OperationalError
from . import constants, logger, config, db, calibre_db, ub, services, get_locale, isoLanguages
from . import logger, config, db, calibre_db, ub, isoLanguages
from .usermanagement import requires_basic_auth_if_no_ano
from .helper import get_download_link, get_book_cover
from .pagination import Pagination
from .web import render_read_books
from .usermanagement import load_user_from_request
from flask_babel import gettext as _
from babel import Locale as LC
from babel.core import UnknownLocaleError
opds = Blueprint('opds', __name__)
log = logger.create()
def requires_basic_auth_if_no_ano(f):
@wraps(f)
def decorated(*args, **kwargs):
auth = request.authorization
if config.config_anonbrowse != 1:
if not auth or auth.type != 'basic' or not check_auth(auth.username, auth.password):
return authenticate()
return f(*args, **kwargs)
if config.config_login_type == constants.LOGIN_LDAP and services.ldap and config.config_anonbrowse != 1:
return services.ldap.basic_auth_required(f)
return decorated
class FeedObject:
def __init__(self, rating_id, rating_name):
self.rating_id = rating_id
self.rating_name = rating_name
@property
def id(self):
return self.rating_id
@property
def name(self):
return self.rating_name
@opds.route("/opds/")
@opds.route("/opds")
@requires_basic_auth_if_no_ano
@ -84,10 +56,12 @@ def feed_osd():
@opds.route("/opds/search", defaults={'query': ""})
@opds.route("/opds/search/<query>")
@opds.route("/opds/search/<path:query>")
@requires_basic_auth_if_no_ano
def feed_cc_search(query):
return feed_search(query.strip())
# Handle strange query from Libera Reader with + instead of spaces
plus_query = unquote_plus(request.environ['RAW_URI'].split('/opds/search/')[1]).strip()
return feed_search(plus_query)
@opds.route("/opds/search", methods=["GET"])
@ -99,26 +73,7 @@ def feed_normal_search():
@opds.route("/opds/books")
@requires_basic_auth_if_no_ano
def feed_booksindex():
shift = 0
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Books.sort, 1, 1)).label('id'))\
.filter(calibre_db.common_filters()).group_by(func.upper(func.substr(db.Books.sort, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_books',
pagination=pagination)
return render_element_index(db.Books.sort, None, 'opds.feed_letter_books')
@opds.route("/opds/books/letter/<book_id>")
@ -129,7 +84,8 @@ def feed_letter_books(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
letter,
[db.Books.sort])
[db.Books.sort],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -139,15 +95,16 @@ def feed_letter_books(book_id):
def feed_new():
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, True, [db.Books.timestamp.desc()])
db.Books, True, [db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/discover")
@requires_basic_auth_if_no_ano
def feed_discover():
entries = calibre_db.session.query(db.Books).filter(calibre_db.common_filters()).order_by(func.random())\
.limit(config.config_books_per_page)
query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
entries = query.filter(calibre_db.common_filters()).order_by(func.random()).limit(config.config_books_per_page)
pagination = Pagination(1, config.config_books_per_page, int(config.config_books_per_page))
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -158,7 +115,8 @@ def feed_best_rated():
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books, db.Books.ratings.any(db.Ratings.rating > 9),
[db.Books.timestamp.desc()])
[db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -171,43 +129,23 @@ def feed_hot():
hot_books = all_books.offset(off).limit(config.config_books_per_page)
entries = list()
for book in hot_books:
downloadBook = calibre_db.get_book(book.Downloads.book_id)
if downloadBook:
entries.append(
calibre_db.get_filtered_book(book.Downloads.book_id)
)
query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
download_book = query.filter(calibre_db.common_filters()).filter(
book.Downloads.book_id == db.Books.id).first()
if download_book:
entries.append(download_book)
else:
ub.delete_download(book.Downloads.book_id)
numBooks = entries.__len__()
num_books = entries.__len__()
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1),
config.config_books_per_page, numBooks)
config.config_books_per_page, num_books)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@opds.route("/opds/author")
@requires_basic_auth_if_no_ano
def feed_authorindex():
shift = 0
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Authors.sort, 1, 1)).label('id'))\
.join(db.books_authors_link).join(db.Books).filter(calibre_db.common_filters())\
.group_by(func.upper(func.substr(db.Authors.sort, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_author',
pagination=pagination)
return render_element_index(db.Authors.sort, db.books_authors_link, 'opds.feed_letter_author')
@opds.route("/opds/author/letter/<book_id>")
@ -228,12 +166,7 @@ def feed_letter_author(book_id):
@opds.route("/opds/author/<int:book_id>")
@requires_basic_auth_if_no_ano
def feed_author(book_id):
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.authors.any(db.Authors.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
return render_xml_dataset(db.Authors, book_id)
@opds.route("/opds/publisher")
@ -254,37 +187,14 @@ def feed_publisherindex():
@opds.route("/opds/publisher/<int:book_id>")
@requires_basic_auth_if_no_ano
def feed_publisher(book_id):
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.publishers.any(db.Publishers.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
return render_xml_dataset(db.Publishers, book_id)
@opds.route("/opds/category")
@requires_basic_auth_if_no_ano
def feed_categoryindex():
shift = 0
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Tags.name, 1, 1)).label('id'))\
.join(db.books_tags_link).join(db.Books).filter(calibre_db.common_filters())\
.group_by(func.upper(func.substr(db.Tags.name, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
return render_element_index(db.Tags.name, db.books_tags_link, 'opds.feed_letter_category')
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_category',
pagination=pagination)
@opds.route("/opds/category/letter/<book_id>")
@requires_basic_auth_if_no_ano
@ -306,36 +216,14 @@ def feed_letter_category(book_id):
@opds.route("/opds/category/<int:book_id>")
@requires_basic_auth_if_no_ano
def feed_category(book_id):
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.tags.any(db.Tags.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
return render_xml_dataset(db.Tags, book_id)
@opds.route("/opds/series")
@requires_basic_auth_if_no_ano
def feed_seriesindex():
shift = 0
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(db.Series.sort, 1, 1)).label('id'))\
.join(db.books_series_link).join(db.Books).filter(calibre_db.common_filters())\
.group_by(func.upper(func.substr(db.Series.sort, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name':_("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder='opds.feed_letter_series',
pagination=pagination)
return render_element_index(db.Series.sort, db.books_series_link, 'opds.feed_letter_series')
@opds.route("/opds/series/letter/<book_id>")
@requires_basic_auth_if_no_ano
@ -361,7 +249,8 @@ def feed_series(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.series.any(db.Series.id == book_id),
[db.Books.series_index])
[db.Books.series_index],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -388,12 +277,7 @@ def feed_ratingindex():
@opds.route("/opds/ratings/<book_id>")
@requires_basic_auth_if_no_ano
def feed_ratings(book_id):
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.ratings.any(db.Ratings.id == book_id),
[db.Books.timestamp.desc()])
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
return render_xml_dataset(db.Ratings, book_id)
@opds.route("/opds/formats")
@ -420,7 +304,8 @@ def feed_format(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.data.any(db.Data.format == book_id.upper()),
[db.Books.timestamp.desc()])
[db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -429,20 +314,12 @@ def feed_format(book_id):
@requires_basic_auth_if_no_ano
def feed_languagesindex():
off = request.args.get("offset") or 0
if current_user.filter_language() == u"all":
if current_user.filter_language() == "all":
languages = calibre_db.speaking_language()
else:
#try:
# cur_l = LC.parse(current_user.filter_language())
#except UnknownLocaleError:
# cur_l = None
languages = calibre_db.session.query(db.Languages).filter(
db.Languages.lang_code == current_user.filter_language()).all()
languages[0].name = isoLanguages.get_language_name(get_locale(), languages[0].lang_code)
#if cur_l:
# languages[0].name = cur_l.get_language_name(get_locale())
#else:
# languages[0].name = _(isoLanguages.get(part3=languages[0].lang_code).name)
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(languages))
return render_xml_template('feed.xml', listelements=languages, folder='opds.feed_languages', pagination=pagination)
@ -455,7 +332,8 @@ def feed_languages(book_id):
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
db.Books.languages.any(db.Languages.id == book_id),
[db.Books.timestamp.desc()])
[db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@ -463,7 +341,8 @@ def feed_languages(book_id):
@requires_basic_auth_if_no_ano
def feed_shelfindex():
off = request.args.get("offset") or 0
shelf = g.shelves_access
shelf = ub.session.query(ub.Shelf).filter(
or_(ub.Shelf.is_public == 1, ub.Shelf.user_id == current_user.id)).order_by(ub.Shelf.name).all()
number = len(shelf)
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
number)
@ -485,24 +364,32 @@ def feed_shelf(book_id):
result = list()
# user is allowed to access shelf
if shelf:
books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == book_id).order_by(
ub.BookShelf.order.asc()).all()
for book in books_in_shelf:
cur_book = calibre_db.get_book(book.book_id)
result.append(cur_book)
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(result))
result, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
config.config_books_per_page,
db.Books,
ub.BookShelf.shelf == shelf.id,
[ub.BookShelf.order.asc()],
True, config.config_read_column,
ub.BookShelf, ub.BookShelf.book_id == db.Books.id)
# delete shelf entries where book is not existent anymore, can happen if book is deleted outside calibre-web
wrong_entries = calibre_db.session.query(ub.BookShelf) \
.join(db.Books, ub.BookShelf.book_id == db.Books.id, isouter=True) \
.filter(db.Books.id == None).all()
for entry in wrong_entries:
log.info('Not existing book {} in {} deleted'.format(entry.book_id, shelf))
try:
ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == entry.book_id).delete()
ub.session.commit()
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error_or_exception("Settings Database error: {}".format(e))
return render_xml_template('feed.xml', entries=result, pagination=pagination)
@opds.route("/opds/download/<book_id>/<book_format>/")
@requires_basic_auth_if_no_ano
def opds_download_link(book_id, book_format):
# I gave up with this: With enabled ldap login, the user doesn't get logged in, therefore it's always guest
# workaround, loading the user from the request and checking it's download rights here
# in case of anonymous browsing user is None
user = load_user_from_request(request) or current_user
if not user.role_download():
if not current_user.role_download():
return abort(403)
if "Kobo" in request.headers.get('User-Agent'):
client = "kobo"
@ -525,47 +412,6 @@ def get_metadata_calibre_companion(uuid, library):
return ""
def feed_search(term):
if term:
entries, __, ___ = calibre_db.get_search_results(term)
entriescount = len(entries) if len(entries) > 0 else 1
pagination = Pagination(1, entriescount, entriescount)
return render_xml_template('feed.xml', searchterm=term, entries=entries, pagination=pagination)
else:
return render_xml_template('feed.xml', searchterm="")
def check_auth(username, password):
try:
username = username.encode('windows-1252')
except UnicodeEncodeError:
username = username.encode('utf-8')
user = ub.session.query(ub.User).filter(func.lower(ub.User.name) ==
username.decode('utf-8').lower()).first()
if bool(user and check_password_hash(str(user.password), password)):
return True
else:
ip_Address = request.headers.get('X-Forwarded-For', request.remote_addr)
log.warning('OPDS Login failed for user "%s" IP-address: %s', username.decode('utf-8'), ip_Address)
return False
def authenticate():
return Response(
'Could not verify your access level for that URL.\n'
'You have to login with proper credentials', 401,
{'WWW-Authenticate': 'Basic realm="Login Required"'})
def render_xml_template(*args, **kwargs):
# ToDo: return time in current timezone similar to %z
currtime = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%S+00:00")
xml = render_template(current_time=currtime, instance=config.config_calibre_web_title, *args, **kwargs)
response = make_response(xml)
response.headers["Content-Type"] = "application/atom+xml; charset=utf-8"
return response
@opds.route("/opds/thumb_240_240/<book_id>")
@opds.route("/opds/cover_240_240/<book_id>")
@opds.route("/opds/cover_90_90/<book_id>")
@ -589,3 +435,71 @@ def feed_unread_books():
off = request.args.get("offset") or 0
result, pagination = render_read_books(int(off) / (int(config.config_books_per_page)) + 1, False, True)
return render_xml_template('feed.xml', entries=result, pagination=pagination)
class FeedObject:
def __init__(self, rating_id, rating_name):
self.rating_id = rating_id
self.rating_name = rating_name
@property
def id(self):
return self.rating_id
@property
def name(self):
return self.rating_name
def feed_search(term):
if term:
entries, __, ___ = calibre_db.get_search_results(term, config=config)
entries_count = len(entries) if len(entries) > 0 else 1
pagination = Pagination(1, entries_count, entries_count)
return render_xml_template('feed.xml', searchterm=term, entries=entries, pagination=pagination)
else:
return render_xml_template('feed.xml', searchterm="")
def render_xml_template(*args, **kwargs):
# ToDo: return time in current timezone similar to %z
currtime = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%S+00:00")
xml = render_template(current_time=currtime, instance=config.config_calibre_web_title, *args, **kwargs)
response = make_response(xml)
response.headers["Content-Type"] = "application/atom+xml; charset=utf-8"
return response
def render_xml_dataset(data_table, book_id):
off = request.args.get("offset") or 0
entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
db.Books,
getattr(db.Books, data_table.__tablename__).any(data_table.id == book_id),
[db.Books.timestamp.desc()],
True, config.config_read_column)
return render_xml_template('feed.xml', entries=entries, pagination=pagination)
def render_element_index(database_column, linked_table, folder):
shift = 0
off = int(request.args.get("offset") or 0)
entries = calibre_db.session.query(func.upper(func.substr(database_column, 1, 1)).label('id'), None, None)
# query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
if linked_table is not None:
entries = entries.join(linked_table).join(db.Books)
entries = entries.filter(calibre_db.common_filters()).group_by(func.upper(func.substr(database_column, 1, 1))).all()
elements = []
if off == 0:
elements.append({'id': "00", 'name': _("All")})
shift = 1
for entry in entries[
off + shift - 1:
int(off + int(config.config_books_per_page) - shift)]:
elements.append({'id': entry.id, 'name': entry.id})
pagination = Pagination((int(off) / (int(config.config_books_per_page)) + 1), config.config_books_per_page,
len(entries) + 1)
return render_xml_template('feed.xml',
letterelements=elements,
folder=folder,
pagination=pagination)

8
cps/pagination.py
View File

@ -57,10 +57,10 @@ class Pagination(object):
def has_next(self):
return self.page < self.pages
# right_edge: last right_edges count of all pages are shown as number, means, if 10 pages are paginated -> 9,10 shwn
# left_edge: first left_edges count of all pages are shown as number -> 1,2 shwn
# left_current: left_current count below current page are shown as number, means if current page 5 -> 3,4 shwn
# left_current: right_current count above current page are shown as number, means if current page 5 -> 6,7 shwn
# right_edge: last right_edges count of all pages are shown as number, means, if 10 pages are paginated -> 9,10 shown
# left_edge: first left_edges count of all pages are shown as number -> 1,2 shown
# left_current: left_current count below current page are shown as number, means if current page 5 -> 3,4 shown
# left_current: right_current count above current page are shown as number, means if current page 5 -> 6,7 shown
def iter_pages(self, left_edge=2, left_current=2,
right_current=4, right_edge=2):
last = 0

7
cps/redirect.py
View File

@ -25,12 +25,9 @@
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE AND DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# http://flask.pocoo.org/snippets/62/
# https://web.archive.org/web/20120517003641/http://flask.pocoo.org/snippets/62/
try:
from urllib.parse import urlparse, urljoin
except ImportError:
from urlparse import urlparse, urljoin
from urllib.parse import urlparse, urljoin
from flask import request, url_for, redirect

29
cps/remotelogin.py
View File

@ -22,6 +22,7 @@
import json
from datetime import datetime
from functools import wraps
from flask import Blueprint, request, make_response, abort, url_for, flash, redirect
from flask_login import login_required, current_user, login_user
@ -31,10 +32,6 @@ from sqlalchemy.sql.expression import true
from . import config, logger, ub
from .render_template import render_title_template
try:
from functools import wraps
except ImportError:
pass # We're not using Python 3
remotelogin = Blueprint('remotelogin', __name__)
log = logger.create()
@ -61,8 +58,8 @@ def remote_login():
ub.session.add(auth_token)
ub.session_commit()
verify_url = url_for('remotelogin.verify_token', token=auth_token.auth_token, _external=true)
log.debug(u"Remot Login request with token: %s", auth_token.auth_token)
return render_title_template('remote_login.html', title=_(u"Login"), token=auth_token.auth_token,
log.debug("Remot Login request with token: %s", auth_token.auth_token)
return render_title_template('remote_login.html', title=_("Login"), token=auth_token.auth_token,
verify_url=verify_url, page="remotelogin")
@ -74,8 +71,8 @@ def verify_token(token):
# Token not found
if auth_token is None:
flash(_(u"Token not found"), category="error")
log.error(u"Remote Login token not found")
flash(_("Token not found"), category="error")
log.error("Remote Login token not found")
return redirect(url_for('web.index'))
# Token expired
@ -83,8 +80,8 @@ def verify_token(token):
ub.session.delete(auth_token)
ub.session_commit()
flash(_(u"Token has expired"), category="error")
log.error(u"Remote Login token expired")
flash(_("Token has expired"), category="error")
log.error("Remote Login token expired")
return redirect(url_for('web.index'))
# Update token with user information
@ -92,8 +89,8 @@ def verify_token(token):
auth_token.verified = True
ub.session_commit()
flash(_(u"Success! Please return to your device"), category="success")
log.debug(u"Remote Login token for userid %s verified", auth_token.user_id)
flash(_("Success! Please return to your device"), category="success")
log.debug("Remote Login token for userid %s verified", auth_token.user_id)
return redirect(url_for('web.index'))
@ -108,7 +105,7 @@ def token_verified():
# Token not found
if auth_token is None:
data['status'] = 'error'
data['message'] = _(u"Token not found")
data['message'] = _("Token not found")
# Token expired
elif datetime.now() > auth_token.expiration:
@ -116,7 +113,7 @@ def token_verified():
ub.session_commit()
data['status'] = 'error'
data['message'] = _(u"Token has expired")
data['message'] = _("Token has expired")
elif not auth_token.verified:
data['status'] = 'not_verified'
@ -129,8 +126,8 @@ def token_verified():
ub.session_commit("User {} logged in via remotelogin, token deleted".format(user.name))
data['status'] = 'success'
log.debug(u"Remote Login for userid %s succeded", user.id)
flash(_(u"you are now logged in as: '%(nickname)s'", nickname=user.name), category="success")
log.debug("Remote Login for userid %s succeeded", user.id)
flash(_("Success! You are now logged in as: %(nickname)s", nickname=user.name), category="success")
response = make_response(json.dumps(data, ensure_ascii=False))
response.headers["Content-Type"] = "application/json; charset=utf-8"

69
cps/render_template.py
View File

@ -16,20 +16,23 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from flask import render_template
from flask import render_template, g, abort, request
from flask_babel import gettext as _
from flask import g
from werkzeug.local import LocalProxy
from flask_login import current_user
from sqlalchemy.sql.expression import or_
from . import config, constants, ub, logger, db, calibre_db
from . import config, constants, logger, ub
from .ub import User
log = logger.create()
def get_sidebar_config(kwargs=None):
kwargs = kwargs or []
simple = bool([e for e in ['kindle', 'tolino', "kobo", "bookeen"]
if (e in request.headers.get('User-Agent', "").lower())])
if 'content' in kwargs:
content = kwargs['content']
content = isinstance(content, (User, LocalProxy)) and not content.role_anonymous()
@ -44,12 +47,12 @@ def get_sidebar_config(kwargs=None):
"show_text": _('Show Hot Books'), "config_show": True})
if current_user.role_admin():
sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.download_list',
"id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not g.user.is_anonymous),
"id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not current_user.is_anonymous),
"page": "download", "show_text": _('Show Downloaded Books'),
"config_show": content})
else:
sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.books_list',
"id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not g.user.is_anonymous),
"id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not current_user.is_anonymous),
"page": "download", "show_text": _('Show Downloaded Books'),
"config_show": content})
sidebar.append(
@ -57,66 +60,60 @@ def get_sidebar_config(kwargs=None):
"visibility": constants.SIDEBAR_BEST_RATED, 'public': True, "page": "rated",
"show_text": _('Show Top Rated Books'), "config_show": True})
sidebar.append({"glyph": "glyphicon-eye-open", "text": _('Read Books'), "link": 'web.books_list', "id": "read",
"visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not g.user.is_anonymous),
"page": "read", "show_text": _('Show read and unread'), "config_show": content})
"visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not current_user.is_anonymous),
"page": "read", "show_text": _('Show Read and Unread'), "config_show": content})
sidebar.append(
{"glyph": "glyphicon-eye-close", "text": _('Unread Books'), "link": 'web.books_list', "id": "unread",
"visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not g.user.is_anonymous), "page": "unread",
"visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not current_user.is_anonymous), "page": "unread",
"show_text": _('Show unread'), "config_show": False})
sidebar.append({"glyph": "glyphicon-random", "text": _('Discover'), "link": 'web.books_list', "id": "rand",
"visibility": constants.SIDEBAR_RANDOM, 'public': True, "page": "discover",
"show_text": _('Show Random Books'), "config_show": True})
sidebar.append({"glyph": "glyphicon-inbox", "text": _('Categories'), "link": 'web.category_list', "id": "cat",
"visibility": constants.SIDEBAR_CATEGORY, 'public': True, "page": "category",
"show_text": _('Show category selection'), "config_show": True})
"show_text": _('Show Category Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-bookmark", "text": _('Series'), "link": 'web.series_list', "id": "serie",
"visibility": constants.SIDEBAR_SERIES, 'public': True, "page": "series",
"show_text": _('Show series selection'), "config_show": True})
"show_text": _('Show Series Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-user", "text": _('Authors'), "link": 'web.author_list', "id": "author",
"visibility": constants.SIDEBAR_AUTHOR, 'public': True, "page": "author",
"show_text": _('Show author selection'), "config_show": True})
"show_text": _('Show Author Section'), "config_show": True})
sidebar.append(
{"glyph": "glyphicon-text-size", "text": _('Publishers'), "link": 'web.publisher_list', "id": "publisher",
"visibility": constants.SIDEBAR_PUBLISHER, 'public': True, "page": "publisher",
"show_text": _('Show publisher selection'), "config_show":True})
"show_text": _('Show Publisher Section'), "config_show":True})
sidebar.append({"glyph": "glyphicon-flag", "text": _('Languages'), "link": 'web.language_overview', "id": "lang",
"visibility": constants.SIDEBAR_LANGUAGE, 'public': (g.user.filter_language() == 'all'),
"visibility": constants.SIDEBAR_LANGUAGE, 'public': (current_user.filter_language() == 'all'),
"page": "language",
"show_text": _('Show language selection'), "config_show": True})
"show_text": _('Show Language Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-star-empty", "text": _('Ratings'), "link": 'web.ratings_list', "id": "rate",
"visibility": constants.SIDEBAR_RATING, 'public': True,
"page": "rating", "show_text": _('Show ratings selection'), "config_show": True})
"page": "rating", "show_text": _('Show Ratings Section'), "config_show": True})
sidebar.append({"glyph": "glyphicon-file", "text": _('File formats'), "link": 'web.formats_list', "id": "format",
"visibility": constants.SIDEBAR_FORMAT, 'public': True,
"page": "format", "show_text": _('Show file formats selection'), "config_show": True})
"page": "format", "show_text": _('Show File Formats Section'), "config_show": True})
sidebar.append(
{"glyph": "glyphicon-trash", "text": _('Archived Books'), "link": 'web.books_list', "id": "archived",
"visibility": constants.SIDEBAR_ARCHIVED, 'public': (not g.user.is_anonymous), "page": "archived",
"show_text": _('Show archived books'), "config_show": content})
"visibility": constants.SIDEBAR_ARCHIVED, 'public': (not current_user.is_anonymous), "page": "archived",
"show_text": _('Show Archived Books'), "config_show": content})
if not simple:
sidebar.append(
{"glyph": "glyphicon-th-list", "text": _('Books List'), "link": 'web.books_table', "id": "list",
"visibility": constants.SIDEBAR_LIST, 'public': (not g.user.is_anonymous), "page": "list",
"visibility": constants.SIDEBAR_LIST, 'public': (not current_user.is_anonymous), "page": "list",
"show_text": _('Show Books List'), "config_show": content})
g.shelves_access = ub.session.query(ub.Shelf).filter(
or_(ub.Shelf.is_public == 1, ub.Shelf.user_id == current_user.id)).order_by(ub.Shelf.name).all()
return sidebar
return sidebar, simple
def get_readbooks_ids():
if not config.config_read_column:
readBooks = ub.session.query(ub.ReadBook).filter(ub.ReadBook.user_id == int(current_user.id))\
.filter(ub.ReadBook.read_status == ub.ReadBook.STATUS_FINISHED).all()
return frozenset([x.book_id for x in readBooks])
else:
try:
readBooks = calibre_db.session.query(db.cc_classes[config.config_read_column])\
.filter(db.cc_classes[config.config_read_column].value == True).all()
return frozenset([x.book for x in readBooks])
except (KeyError, AttributeError):
log.error("Custom Column No.%d is not existing in calibre database", config.config_read_column)
return []
# Returns the template for rendering and includes the instance name
def render_title_template(*args, **kwargs):
sidebar = get_sidebar_config(kwargs)
return render_template(instance=config.config_calibre_web_title, sidebar=sidebar,
accept=constants.EXTENSIONS_UPLOAD, read_book_ids=get_readbooks_ids(),
sidebar, simple = get_sidebar_config(kwargs)
try:
return render_template(instance=config.config_calibre_web_title, sidebar=sidebar, simple=simple,
accept=constants.EXTENSIONS_UPLOAD,
*args, **kwargs)
except PermissionError:
log.error("No permission to access {} file.".format(args[0]))
abort(403)

102
cps/schedule.py Normal file
View File

@ -0,0 +1,102 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2020 mmonkey
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import datetime
from . import config, constants
from .services.background_scheduler import BackgroundScheduler, CronTrigger, use_APScheduler
from .tasks.database import TaskReconnectDatabase
from .tasks.thumbnail import TaskGenerateCoverThumbnails, TaskGenerateSeriesThumbnails, TaskClearCoverThumbnailCache
from .services.worker import WorkerThread
from .tasks.metadata_backup import TaskBackupMetadata
def get_scheduled_tasks(reconnect=True):
tasks = list()
# config.schedule_reconnect or
# Reconnect Calibre database (metadata.db)
if reconnect:
tasks.append([lambda: TaskReconnectDatabase(), 'reconnect', False])
# ToDo make configurable. Generate metadata.opf file for each changed book
if False:
tasks.append([lambda: TaskBackupMetadata("en"), 'backup metadata', False])
# Generate all missing book cover thumbnails
if config.schedule_generate_book_covers:
tasks.append([lambda: TaskClearCoverThumbnailCache(0), 'delete superfluous book covers', True])
tasks.append([lambda: TaskGenerateCoverThumbnails(), 'generate book covers', False])
# Generate all missing series thumbnails
if config.schedule_generate_series_covers:
tasks.append([lambda: TaskGenerateSeriesThumbnails(), 'generate book covers', False])
return tasks
def end_scheduled_tasks():
worker = WorkerThread.get_instance()
for __, __, __, task, __ in worker.tasks:
if task.scheduled and task.is_cancellable:
worker.end_task(task.id)
def register_scheduled_tasks(reconnect=True):
scheduler = BackgroundScheduler()
if scheduler:
# Remove all existing jobs
scheduler.remove_all_jobs()
start = config.schedule_start_time
duration = config.schedule_duration
# Register scheduled tasks
scheduler.schedule_tasks(tasks=get_scheduled_tasks(reconnect), trigger=CronTrigger(hour=start))
end_time = calclulate_end_time(start, duration)
scheduler.schedule(func=end_scheduled_tasks, trigger=CronTrigger(hour=end_time.hour, minute=end_time.minute),
name="end scheduled task")
# Kick-off tasks, if they should currently be running
if should_task_be_running(start, duration):
scheduler.schedule_tasks_immediately(tasks=get_scheduled_tasks(reconnect))
def register_startup_tasks():
scheduler = BackgroundScheduler()
if scheduler:
start = config.schedule_start_time
duration = config.schedule_duration
# Run scheduled tasks immediately for development and testing
# Ignore tasks that should currently be running, as these will be added when registering scheduled tasks
if constants.APP_MODE in ['development', 'test'] and not should_task_be_running(start, duration):
scheduler.schedule_tasks_immediately(tasks=get_scheduled_tasks(False))
def should_task_be_running(start, duration):
now = datetime.datetime.now()
start_time = datetime.datetime.now().replace(hour=start, minute=0, second=0, microsecond=0)
end_time = start_time + datetime.timedelta(hours=duration // 60, minutes=duration % 60)
return start_time < now < end_time
def calclulate_end_time(start, duration):
start_time = datetime.datetime.now().replace(hour=start, minute=0)
return start_time + datetime.timedelta(hours=duration // 60, minutes=duration % 60)

397
cps/search.py Normal file
View File

@ -0,0 +1,397 @@
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2022 OzzieIsaacs
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import json
from datetime import datetime
from flask import Blueprint, request, redirect, url_for, flash
from flask import session as flask_session
from flask_login import current_user
from flask_babel import format_date
from flask_babel import gettext as _
from sqlalchemy.sql.expression import func, not_, and_, or_, text, true
from sqlalchemy.sql.functions import coalesce
from . import logger, db, calibre_db, config, ub
from .usermanagement import login_required_if_no_ano
from .render_template import render_title_template
from .pagination import Pagination
search = Blueprint('search', __name__)
log = logger.create()
@search.route("/search", methods=["POST"])
@login_required_if_no_ano
def simple_search():
term = dict(request.form).get("query")
if term:
flask_session['query'] = json.dumps(term.strip())
return redirect(url_for('web.books_list', data="search", sort_param='stored', query="")) # term.strip()
else:
return render_title_template('search.html',
searchterm="",
result_count=0,
title=_("Search"),
page="search")
@search.route("/advsearch", methods=['POST'])
@login_required_if_no_ano
def advanced_search():
values = dict(request.form)
params = ['include_tag', 'exclude_tag', 'include_serie', 'exclude_serie', 'include_shelf', 'exclude_shelf',
'include_language', 'exclude_language', 'include_extension', 'exclude_extension']
for param in params:
values[param] = list(request.form.getlist(param))
flask_session['query'] = json.dumps(values)
return redirect(url_for('web.books_list', data="advsearch", sort_param='stored', query=""))
@search.route("/advsearch", methods=['GET'])
@login_required_if_no_ano
def advanced_search_form():
# Build custom columns names
cc = calibre_db.get_cc_columns(config, filter_config_custom_read=True)
return render_prepare_search_form(cc)
def adv_search_custom_columns(cc, term, q):
for c in cc:
if c.datatype == "datetime":
custom_start = term.get('custom_column_' + str(c.id) + '_start')
custom_end = term.get('custom_column_' + str(c.id) + '_end')
if custom_start:
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
func.datetime(db.cc_classes[c.id].value) >= func.datetime(custom_start)))
if custom_end:
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
func.datetime(db.cc_classes[c.id].value) <= func.datetime(custom_end)))
else:
custom_query = term.get('custom_column_' + str(c.id))
if custom_query != '' and custom_query is not None:
if c.datatype == 'bool':
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
db.cc_classes[c.id].value == (custom_query == "True")))
elif c.datatype == 'int' or c.datatype == 'float':
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
db.cc_classes[c.id].value == custom_query))
elif c.datatype == 'rating':
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
db.cc_classes[c.id].value == int(float(custom_query) * 2)))
else:
q = q.filter(getattr(db.Books, 'custom_column_' + str(c.id)).any(
func.lower(db.cc_classes[c.id].value).ilike("%" + custom_query + "%")))
return q
def adv_search_language(q, include_languages_inputs, exclude_languages_inputs):
if current_user.filter_language() != "all":
q = q.filter(db.Books.languages.any(db.Languages.lang_code == current_user.filter_language()))
else:
for language in include_languages_inputs:
q = q.filter(db.Books.languages.any(db.Languages.id == language))
for language in exclude_languages_inputs:
q = q.filter(not_(db.Books.series.any(db.Languages.id == language)))
return q
def adv_search_ratings(q, rating_high, rating_low):
if rating_high:
rating_high = int(rating_high) * 2
q = q.filter(db.Books.ratings.any(db.Ratings.rating <= rating_high))
if rating_low:
rating_low = int(rating_low) * 2
q = q.filter(db.Books.ratings.any(db.Ratings.rating >= rating_low))
return q
def adv_search_read_status(read_status):
if not config.config_read_column:
if read_status == "True":
db_filter = and_(ub.ReadBook.user_id == int(current_user.id),
ub.ReadBook.read_status == ub.ReadBook.STATUS_FINISHED)
else:
db_filter = coalesce(ub.ReadBook.read_status, 0) != ub.ReadBook.STATUS_FINISHED
else:
try:
if read_status == "True":
db_filter = db.cc_classes[config.config_read_column].value == True
else:
db_filter = coalesce(db.cc_classes[config.config_read_column].value, False) != True
except (KeyError, AttributeError, IndexError):
log.error("Custom Column No.{} does not exist in calibre database".format(config.config_read_column))
flash(_("Custom Column No.%(column)d does not exist in calibre database",
column=config.config_read_column),
category="error")
return true()
return db_filter
def adv_search_extension(q, include_extension_inputs, exclude_extension_inputs):
for extension in include_extension_inputs:
q = q.filter(db.Books.data.any(db.Data.format == extension))
for extension in exclude_extension_inputs:
q = q.filter(not_(db.Books.data.any(db.Data.format == extension)))
return q
def adv_search_tag(q, include_tag_inputs, exclude_tag_inputs):
for tag in include_tag_inputs:
q = q.filter(db.Books.tags.any(db.Tags.id == tag))
for tag in exclude_tag_inputs:
q = q.filter(not_(db.Books.tags.any(db.Tags.id == tag)))
return q
def adv_search_serie(q, include_series_inputs, exclude_series_inputs):
for serie in include_series_inputs:
q = q.filter(db.Books.series.any(db.Series.id == serie))
for serie in exclude_series_inputs:
q = q.filter(not_(db.Books.series.any(db.Series.id == serie)))
return q
def adv_search_shelf(q, include_shelf_inputs, exclude_shelf_inputs):
q = q.outerjoin(ub.BookShelf, db.Books.id == ub.BookShelf.book_id)\
.filter(or_(ub.BookShelf.shelf == None, ub.BookShelf.shelf.notin_(exclude_shelf_inputs)))
if len(include_shelf_inputs) > 0:
q = q.filter(ub.BookShelf.shelf.in_(include_shelf_inputs))
return q
def extend_search_term(searchterm,
author_name,
book_title,
publisher,
pub_start,
pub_end,
tags,
rating_high,
rating_low,
read_status,
):
searchterm.extend((author_name.replace('|', ','), book_title, publisher))
if pub_start:
try:
searchterm.extend([_("Published after ") +
format_date(datetime.strptime(pub_start, "%Y-%m-%d"),
format='medium')])
except ValueError:
pub_start = ""
if pub_end:
try:
searchterm.extend([_("Published before ") +
format_date(datetime.strptime(pub_end, "%Y-%m-%d"),
format='medium')])
except ValueError:
pub_end = ""
elements = {'tag': db.Tags, 'serie':db.Series, 'shelf':ub.Shelf}
for key, db_element in elements.items():
tag_names = calibre_db.session.query(db_element).filter(db_element.id.in_(tags['include_' + key])).all()
searchterm.extend(tag.name for tag in tag_names)
tag_names = calibre_db.session.query(db_element).filter(db_element.id.in_(tags['exclude_' + key])).all()
searchterm.extend(tag.name for tag in tag_names)
language_names = calibre_db.session.query(db.Languages). \
filter(db.Languages.id.in_(tags['include_language'])).all()
if language_names:
language_names = calibre_db.speaking_language(language_names)
searchterm.extend(language.name for language in language_names)
language_names = calibre_db.session.query(db.Languages). \
filter(db.Languages.id.in_(tags['exclude_language'])).all()
if language_names:
language_names = calibre_db.speaking_language(language_names)
searchterm.extend(language.name for language in language_names)
if rating_high:
searchterm.extend([_("Rating <= %(rating)s", rating=rating_high)])
if rating_low:
searchterm.extend([_("Rating >= %(rating)s", rating=rating_low)])
if read_status:
searchterm.extend([_("Read Status = %(status)s", status=read_status)])
searchterm.extend(ext for ext in tags['include_extension'])
searchterm.extend(ext for ext in tags['exclude_extension'])
# handle custom columns
searchterm = " + ".join(filter(None, searchterm))
return searchterm, pub_start, pub_end
def render_adv_search_results(term, offset=None, order=None, limit=None):
sort = order[0] if order else [db.Books.sort]
pagination = None
cc = calibre_db.get_cc_columns(config, filter_config_custom_read=True)
calibre_db.session.connection().connection.connection.create_function("lower", 1, db.lcase)
query = calibre_db.generate_linked_query(config.config_read_column, db.Books)
q = query.outerjoin(db.books_series_link, db.Books.id == db.books_series_link.c.book)\
.outerjoin(db.Series)\
.filter(calibre_db.common_filters(True))
# parse multi selects to a complete dict
tags = dict()
elements = ['tag', 'serie', 'shelf', 'language', 'extension']
for element in elements:
tags['include_' + element] = term.get('include_' + element)
tags['exclude_' + element] = term.get('exclude_' + element)
author_name = term.get("author_name")
book_title = term.get("book_title")
publisher = term.get("publisher")
pub_start = term.get("publishstart")
pub_end = term.get("publishend")
rating_low = term.get("ratinghigh")
rating_high = term.get("ratinglow")
description = term.get("comment")
read_status = term.get("read_status")
if author_name:
author_name = author_name.strip().lower().replace(',', '|')
if book_title:
book_title = book_title.strip().lower()
if publisher:
publisher = publisher.strip().lower()
search_term = []
cc_present = False
for c in cc:
if c.datatype == "datetime":
column_start = term.get('custom_column_' + str(c.id) + '_start')
column_end = term.get('custom_column_' + str(c.id) + '_end')
if column_start:
search_term.extend(["{} >= {}".format(c.name,
format_date(datetime.strptime(column_start, "%Y-%m-%d").date(),
format='medium')
)])
cc_present = True
if column_end:
search_term.extend(["{} <= {}".format(c.name,
format_date(datetime.strptime(column_end, "%Y-%m-%d").date(),
format='medium')
)])
cc_present = True
elif term.get('custom_column_' + str(c.id)):
search_term.extend([("{}: {}".format(c.name, term.get('custom_column_' + str(c.id))))])
cc_present = True
if any(tags.values()) or author_name or book_title or publisher or pub_start or pub_end or rating_low \
or rating_high or description or cc_present or read_status:
search_term, pub_start, pub_end = extend_search_term(search_term,
author_name,
book_title,
publisher,
pub_start,
pub_end,
tags,
rating_high,
rating_low,
read_status)
if author_name:
q = q.filter(db.Books.authors.any(func.lower(db.Authors.name).ilike("%" + author_name + "%")))
if book_title:
q = q.filter(func.lower(db.Books.title).ilike("%" + book_title + "%"))
if pub_start:
q = q.filter(func.datetime(db.Books.pubdate) > func.datetime(pub_start))
if pub_end:
q = q.filter(func.datetime(db.Books.pubdate) < func.datetime(pub_end))
q = q.filter(adv_search_read_status(read_status))
if publisher:
q = q.filter(db.Books.publishers.any(func.lower(db.Publishers.name).ilike("%" + publisher + "%")))
q = adv_search_tag(q, tags['include_tag'], tags['exclude_tag'])
q = adv_search_serie(q, tags['include_serie'], tags['exclude_serie'])
q = adv_search_shelf(q, tags['include_shelf'], tags['exclude_shelf'])
q = adv_search_extension(q, tags['include_extension'], tags['exclude_extension'])
q = adv_search_language(q, tags['include_language'], tags['exclude_language'])
q = adv_search_ratings(q, rating_high, rating_low)
if description:
q = q.filter(db.Books.comments.any(func.lower(db.Comments.text).ilike("%" + description + "%")))
# search custom columns
try:
q = adv_search_custom_columns(cc, term, q)
except AttributeError as ex:
log.debug_or_exception(ex)
flash(_("Error on search for custom columns, please restart Calibre-Web"), category="error")
q = q.order_by(*sort).all()
flask_session['query'] = json.dumps(term)
ub.store_combo_ids(q)
result_count = len(q)
if offset is not None and limit is not None:
offset = int(offset)
limit_all = offset + int(limit)
pagination = Pagination((offset / (int(limit)) + 1), limit, result_count)
else:
offset = 0
limit_all = result_count
entries = calibre_db.order_authors(q[offset:limit_all], list_return=True, combined=True)
return render_title_template('search.html',
adv_searchterm=search_term,
pagination=pagination,
entries=entries,
result_count=result_count,
title=_("Advanced Search"), page="advsearch",
order=order[1])
def render_prepare_search_form(cc):
# prepare data for search-form
tags = calibre_db.session.query(db.Tags)\
.join(db.books_tags_link)\
.join(db.Books)\
.filter(calibre_db.common_filters()) \
.group_by(text('books_tags_link.tag'))\
.order_by(db.Tags.name).all()
series = calibre_db.session.query(db.Series)\
.join(db.books_series_link)\
.join(db.Books)\
.filter(calibre_db.common_filters()) \
.group_by(text('books_series_link.series'))\
.order_by(db.Series.name)\
.filter(calibre_db.common_filters()).all()
shelves = ub.session.query(ub.Shelf)\
.filter(or_(ub.Shelf.is_public == 1, ub.Shelf.user_id == int(current_user.id)))\
.order_by(ub.Shelf.name).all()
extensions = calibre_db.session.query(db.Data)\
.join(db.Books)\
.filter(calibre_db.common_filters()) \
.group_by(db.Data.format)\
.order_by(db.Data.format).all()
if current_user.filter_language() == "all":
languages = calibre_db.speaking_language()
else:
languages = None
return render_title_template('search_form.html', tags=tags, languages=languages, extensions=extensions,
series=series,shelves=shelves, title=_("Advanced Search"), cc=cc, page="advsearch")
def render_search_results(term, offset=None, order=None, limit=None):
join = db.books_series_link, db.Books.id == db.books_series_link.c.book, db.Series
entries, result_count, pagination = calibre_db.get_search_results(term,
config,
offset,
order,
limit,
*join)
return render_title_template('search.html',
searchterm=term,
pagination=pagination,
query=term,
adv_searchterm=term,
entries=entries,
result_count=result_count,
title=_("Search"),
page="search",
order=order[1])

110
cps/search_metadata.py
View File

@ -16,69 +16,91 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
import json
import importlib
import sys
import inspect
import datetime
import concurrent.futures
import importlib
import inspect
import json
import os
import sys
from flask import Blueprint, request, Response, url_for
from flask import Blueprint, Response, request, url_for
from flask_login import current_user
from flask_login import login_required
from flask_babel import get_locale
from sqlalchemy.exc import InvalidRequestError, OperationalError
from sqlalchemy.orm.attributes import flag_modified
from sqlalchemy.exc import OperationalError, InvalidRequestError
from . import constants, logger, ub
from cps.services.Metadata import Metadata
from . import constants, logger, ub, web_server
# current_milli_time = lambda: int(round(time() * 1000))
meta = Blueprint('metadata', __name__)
meta = Blueprint("metadata", __name__)
log = logger.create()
try:
from dataclasses import asdict
except ImportError:
log.info('*** "dataclasses" is needed for calibre-web to run. Please install it using pip: "pip install dataclasses" ***')
print('*** "dataclasses" is needed for calibre-web to run. Please install it using pip: "pip install dataclasses" ***')
web_server.stop(True)
sys.exit(6)
new_list = list()
meta_dir = os.path.join(constants.BASE_DIR, "cps", "metadata_provider")
modules = os.listdir(os.path.join(constants.BASE_DIR, "cps", "metadata_provider"))
for f in modules:
if os.path.isfile(os.path.join(meta_dir, f)) and not f.endswith('__init__.py'):
if os.path.isfile(os.path.join(meta_dir, f)) and not f.endswith("__init__.py"):
a = os.path.basename(f)[:-3]
try:
importlib.import_module("cps.metadata_provider." + a)
new_list.append(a)
except ImportError:
log.error("Import error for metadata source: {}".format(a))
pass
except (IndentationError, SyntaxError) as e:
log.error("Syntax error for metadata source: {} - {}".format(a, e))
except ImportError as e:
log.debug("Import error for metadata source: {} - {}".format(a, e))
def list_classes(provider_list):
classes = list()
for element in provider_list:
for name, obj in inspect.getmembers(sys.modules["cps.metadata_provider." + element]):
if inspect.isclass(obj) and name != "Metadata" and issubclass(obj, Metadata):
for name, obj in inspect.getmembers(
sys.modules["cps.metadata_provider." + element]
):
if (
inspect.isclass(obj)
and name != "Metadata"
and issubclass(obj, Metadata)
):
classes.append(obj())
return classes
cl = list_classes(new_list)
@meta.route("/metadata/provider")
@login_required
def metadata_provider():
active = current_user.view_settings.get('metadata', {})
active = current_user.view_settings.get("metadata", {})
provider = list()
for c in cl:
ac = active.get(c.__id__, True)
provider.append({"name": c.__name__, "active": ac, "initial": ac, "id": c.__id__})
return Response(json.dumps(provider), mimetype='application/json')
provider.append(
{"name": c.__name__, "active": ac, "initial": ac, "id": c.__id__}
)
return Response(json.dumps(provider), mimetype="application/json")
@meta.route("/metadata/provider", methods=['POST'])
@meta.route("/metadata/provider/<prov_name>", methods=['POST'])
@meta.route("/metadata/provider", methods=["POST"])
@meta.route("/metadata/provider/<prov_name>", methods=["POST"])
@login_required
def metadata_change_active_provider(prov_name):
new_state = request.get_json()
active = current_user.view_settings.get('metadata', {})
active[new_state['id']] = new_state['value']
current_user.view_settings['metadata'] = active
active = current_user.view_settings.get("metadata", {})
active[new_state["id"]] = new_state["value"]
current_user.view_settings["metadata"] = active
try:
try:
flag_modified(current_user, "view_settings")
@ -89,29 +111,33 @@ def metadata_change_active_provider(prov_name):
log.error("Invalid request received: {}".format(request))
return "Invalid request", 400
if "initial" in new_state and prov_name:
for c in cl:
if c.__id__ == prov_name:
data = c.search(new_state.get('query', ""))
break
return Response(json.dumps(data), mimetype='application/json')
data = []
provider = next((c for c in cl if c.__id__ == prov_name), None)
if provider is not None:
data = provider.search(new_state.get("query", ""))
return Response(
json.dumps([asdict(x) for x in data]), mimetype="application/json"
)
return ""
@meta.route("/metadata/search", methods=['POST'])
@meta.route("/metadata/search", methods=["POST"])
@login_required
def metadata_search():
query = request.form.to_dict().get('query')
query = request.form.to_dict().get("query")
data = list()
active = current_user.view_settings.get('metadata', {})
active = current_user.view_settings.get("metadata", {})
locale = get_locale()
if query:
static_cover = url_for('static', filename='generic_cover.jpg')
static_cover = url_for("static", filename="generic_cover.jpg")
# start = current_milli_time()
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
meta = {executor.submit(c.search, query, static_cover): c for c in cl if active.get(c.__id__, True)}
meta = {
executor.submit(c.search, query, static_cover, locale): c
for c in cl
if active.get(c.__id__, True)
}
for future in concurrent.futures.as_completed(meta):
data.extend(future.result())
return Response(json.dumps(data), mimetype='application/json')
data.extend([asdict(x) for x in future.result() if x])
# log.info({'Time elapsed {}'.format(current_milli_time()-start)})
return Response(json.dumps(data), mimetype="application/json")

19
cps/server.py
View File

@ -22,9 +22,11 @@ import errno
import signal
import socket
import subprocess # nosec
from .services.background_scheduler import BackgroundScheduler
try:
from gevent.pywsgi import WSGIServer
from .gevent_wsgi import MyWSGIHandler
from gevent.pool import Pool
from gevent import __version__ as _version
from greenlet import GreenletExit
@ -32,7 +34,7 @@ try:
VERSION = 'Gevent ' + _version
_GEVENT = True
except ImportError:
from tornado.wsgi import WSGIContainer
from .tornado_wsgi import MyWSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from tornado import version as _version
@ -202,7 +204,9 @@ class WebServer(object):
if output is None:
output = _readable_listen_address(self.listen_address, self.listen_port)
log.info('Starting Gevent server on %s', output)
self.wsgiserver = WSGIServer(sock, self.app, log=self.access_logger, spawn=Pool(), **ssl_args)
self.wsgiserver = WSGIServer(sock, self.app, log=self.access_logger, handler_class=MyWSGIHandler,
error_log=log,
spawn=Pool(), **ssl_args)
if ssl_args:
wrap_socket = self.wsgiserver.wrap_socket
def my_wrap_socket(*args, **kwargs):
@ -225,8 +229,8 @@ class WebServer(object):
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
log.info('Starting Tornado server on %s', _readable_listen_address(self.listen_address, self.listen_port))
# Max Buffersize set to 200MB )
http_server = HTTPServer(WSGIContainer(self.app),
# Max Buffersize set to 200MB
http_server = HTTPServer(MyWSGIContainer(self.app),
max_buffer_size=209700000,
ssl_options=self.ssl_args)
http_server.listen(self.listen_port, self.listen_address)
@ -262,6 +266,12 @@ class WebServer(object):
subprocess.call(args, close_fds=True) # nosec
return True
@staticmethod
def shutdown_scheduler():
scheduler = BackgroundScheduler()
if scheduler:
scheduler.scheduler.shutdown()
def _killServer(self, __, ___):
self.stop()
@ -270,6 +280,7 @@ class WebServer(object):
updater_thread.stop()
log.info("webserver stop (restart=%s)", restart)
self.shutdown_scheduler()
self.restart = restart
if self.wsgiserver:
if _GEVENT:

86
cps/services/Metadata.py
View File

@ -15,13 +15,97 @@
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import abc
import dataclasses
import os
import re
from typing import Dict, Generator, List, Optional, Union
from cps import constants
class Metadata():
@dataclasses.dataclass
class MetaSourceInfo:
id: str
description: str
link: str
@dataclasses.dataclass
class MetaRecord:
id: Union[str, int]
title: str
authors: List[str]
url: str
source: MetaSourceInfo
cover: str = os.path.join(constants.STATIC_DIR, 'generic_cover.jpg')
description: Optional[str] = ""
series: Optional[str] = None
series_index: Optional[Union[int, float]] = 0
identifiers: Dict[str, Union[str, int]] = dataclasses.field(default_factory=dict)
publisher: Optional[str] = None
publishedDate: Optional[str] = None
rating: Optional[int] = 0
languages: Optional[List[str]] = dataclasses.field(default_factory=list)
tags: Optional[List[str]] = dataclasses.field(default_factory=list)
class Metadata:
__name__ = "Generic"
__id__ = "generic"
def __init__(self):
self.active = True
def set_status(self, state):
self.active = state
@abc.abstractmethod
def search(
self, query: str, generic_cover: str = "", locale: str = "en"
) -> Optional[List[MetaRecord]]:
pass
@staticmethod
def get_title_tokens(
title: str, strip_joiners: bool = True
) -> Generator[str, None, None]:
"""
Taken from calibre source code
It's a simplified (cut out what is unnecessary) version of
https://github.com/kovidgoyal/calibre/blob/99d85b97918625d172227c8ffb7e0c71794966c0/
src/calibre/ebooks/metadata/sources/base.py#L363-L367
(src/calibre/ebooks/metadata/sources/base.py - lines 363-398)
"""
title_patterns = [
(re.compile(pat, re.IGNORECASE), repl)
for pat, repl in [
# Remove things like: (2010) (Omnibus) etc.
(
r"(?i)[({\[](\d{4}|omnibus|anthology|hardcover|"
r"audiobook|audio\scd|paperback|turtleback|"
r"mass\s*market|edition|ed\.)[\])}]",
"",
),
# Remove any strings that contain the substring edition inside
# parentheses
(r"(?i)[({\[].*?(edition|ed.).*?[\]})]", ""),
# Remove commas used a separators in numbers
(r"(\d+),(\d+)", r"\1\2"),
# Remove hyphens only if they have whitespace before them
(r"(\s-)", " "),
# Replace other special chars with a space
(r"""[:,;!@$%^&*(){}.`~"\s\[\]/]《》「」“”""", " "),
]
]
for pat, repl in title_patterns:
title = pat.sub(repl, title)
tokens = title.split()
for token in tokens:
token = token.strip().strip('"').strip("'")
if token and (
not strip_joiners or token.lower() not in ("a", "and", "the", "&")
):
yield token

19
cps/services/SyncToken.py
View File

@ -21,11 +21,8 @@ import sys
from base64 import b64decode, b64encode
from jsonschema import validate, exceptions, __version__
from datetime import datetime
try:
# pylint: disable=unused-import
from urllib import unquote
except ImportError:
from urllib.parse import unquote
from urllib.parse import unquote
from flask import json
from .. import logger
@ -138,12 +135,9 @@ class SyncToken:
archive_last_modified = get_datetime_from_json(data_json, "archive_last_modified")
reading_state_last_modified = get_datetime_from_json(data_json, "reading_state_last_modified")
tags_last_modified = get_datetime_from_json(data_json, "tags_last_modified")
# books_last_id = data_json["books_last_id"]
except TypeError:
log.error("SyncToken timestamps don't parse to a datetime.")
return SyncToken(raw_kobo_store_token=raw_kobo_store_token)
#except KeyError:
# books_last_id = -1
return SyncToken(
raw_kobo_store_token=raw_kobo_store_token,
@ -152,7 +146,6 @@ class SyncToken:
archive_last_modified=archive_last_modified,
reading_state_last_modified=reading_state_last_modified,
tags_last_modified=tags_last_modified,
#books_last_id=books_last_id
)
def set_kobo_store_header(self, store_headers):
@ -176,16 +169,14 @@ class SyncToken:
"archive_last_modified": to_epoch_timestamp(self.archive_last_modified),
"reading_state_last_modified": to_epoch_timestamp(self.reading_state_last_modified),
"tags_last_modified": to_epoch_timestamp(self.tags_last_modified),
#"books_last_id":self.books_last_id
},
}
return b64encode_json(token)
def __str__(self):
return "{},{},{},{},{},{}".format(self.raw_kobo_store_token,
self.books_last_created,
return "{},{},{},{},{},{}".format(self.books_last_created,
self.books_last_modified,
self.archive_last_modified,
self.reading_state_last_modified,
self.tags_last_modified)
#self.books_last_id)
self.tags_last_modified,
self.raw_kobo_store_token)

5
cps/services/__init__.py
View File

@ -18,11 +18,10 @@
from .. import logger
log = logger.create()
try: from . import goodreads_support
try:
from . import goodreads_support
except ImportError as err:
log.debug("Cannot import goodreads, showing authors-metadata will not work: %s", err)
goodreads_support = None

84
cps/services/background_scheduler.py Normal file
View File

@ -0,0 +1,84 @@
# -*- coding: utf-8 -*-
# This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
# Copyright (C) 2020 mmonkey
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import atexit
from .. import logger
from .worker import WorkerThread
try:
from apscheduler.schedulers.background import BackgroundScheduler as BScheduler
from apscheduler.triggers.cron import CronTrigger
from apscheduler.triggers.date import DateTrigger
use_APScheduler = True
except (ImportError, RuntimeError) as e:
use_APScheduler = False
log = logger.create()
log.info('APScheduler not found. Unable to schedule tasks.')
class BackgroundScheduler:
_instance = None
def __new__(cls):
if not use_APScheduler:
return False
if cls._instance is None:
cls._instance = super(BackgroundScheduler, cls).__new__(cls)
cls.log = logger.create()
cls.scheduler = BScheduler()
cls.scheduler.start()
return cls._instance
def schedule(self, func, trigger, name=None):
if use_APScheduler:
return self.scheduler.add_job(func=func, trigger=trigger, name=name)
# Expects a lambda expression for the task
def schedule_task(self, task, user=None, name=None, hidden=False, trigger=None):
if use_APScheduler:
def scheduled_task():
worker_task = task()
worker_task.scheduled = True
WorkerThread.add(user, worker_task, hidden=hidden)
return self.schedule(func=scheduled_task, trigger=trigger, name=name)
# Expects a list of lambda expressions for the tasks
def schedule_tasks(self, tasks, user=None, trigger=None):
if use_APScheduler:
for task in tasks:
self.schedule_task(task[0], user=user, trigger=trigger, name=task[1], hidden=task[2])
# Expects a lambda expression for the task
def schedule_task_immediately(self, task, user=None, name=None, hidden=False):
if use_APScheduler:
def immediate_task():
WorkerThread.add(user, task(), hidden)
return self.schedule(func=immediate_task, trigger=DateTrigger(), name=name)
# Expects a list of lambda expressions for the tasks
def schedule_tasks_immediately(self, tasks, user=None):
if use_APScheduler:
for task in tasks:
self.schedule_task_immediately(task[0], user, name="immediately " + task[1], hidden=task[2])
# Remove all jobs
def remove_all_jobs(self):
self.scheduler.remove_all_jobs()

6
cps/services/gmail.py
View File

@ -25,7 +25,7 @@ from google.oauth2.credentials import Credentials
from datetime import datetime
import base64
from flask_babel import gettext as _
from ..constants import BASE_DIR
from ..constants import CONFIG_DIR
from .. import logger
@ -53,11 +53,11 @@ def setup_gmail(token):
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
cred_file = os.path.join(BASE_DIR, 'gmail.json')
cred_file = os.path.join(CONFIG_DIR, 'gmail.json')
if not os.path.exists(cred_file):
raise Exception(_("Found no valid gmail.json file with OAuth information"))
flow = InstalledAppFlow.from_client_secrets_file(
os.path.join(BASE_DIR, 'gmail.json'), SCOPES)
os.path.join(CONFIG_DIR, 'gmail.json'), SCOPES)
creds = flow.run_local_server(port=0)
user_info = get_user_info(creds)
return {

71
cps/services/worker.py
View File

@ -37,11 +37,13 @@ STAT_WAITING = 0
STAT_FAIL = 1
STAT_STARTED = 2
STAT_FINISH_SUCCESS = 3
STAT_ENDED = 4
STAT_CANCELLED = 5
# Only retain this many tasks in dequeued list
TASK_CLEANUP_TRIGGER = 20
QueuedTask = namedtuple('QueuedTask', 'num, user, added, task')
QueuedTask = namedtuple('QueuedTask', 'num, user, added, task, hidden')
def _get_main_thread():
@ -51,7 +53,6 @@ def _get_main_thread():
raise Exception("main thread not found?!")
class ImprovedQueue(queue.Queue):
def to_list(self):
"""
@ -61,12 +62,13 @@ class ImprovedQueue(queue.Queue):
with self.mutex:
return list(self.queue)
# Class for all worker tasks in the background
class WorkerThread(threading.Thread):
_instance = None
@classmethod
def getInstance(cls):
def get_instance(cls):
if cls._instance is None:
cls._instance = WorkerThread()
return cls._instance
@ -82,15 +84,17 @@ class WorkerThread(threading.Thread):
self.start()
@classmethod
def add(cls, user, task):
ins = cls.getInstance()
def add(cls, user, task, hidden=False):
ins = cls.get_instance()
ins.num += 1
log.debug("Add Task for user: {}: {}".format(user, task))
username = user if user is not None else 'System'
log.debug("Add Task for user: {} - {}".format(username, task))
ins.queue.put(QueuedTask(
num=ins.num,
user=user,
user=username,
added=datetime.now(),
task=task,
hidden=hidden
))
@property
@ -111,10 +115,10 @@ class WorkerThread(threading.Thread):
if delta > TASK_CLEANUP_TRIGGER:
ret = alive
else:
# otherwise, lop off the oldest dead tasks until we hit the target trigger
ret = sorted(dead, key=lambda x: x.task.end_time)[-TASK_CLEANUP_TRIGGER:] + alive
# otherwise, loop off the oldest dead tasks until we hit the target trigger
ret = sorted(dead, key=lambda y: y.task.end_time)[-TASK_CLEANUP_TRIGGER:] + alive
self.dequeued = sorted(ret, key=lambda x: x.num)
self.dequeued = sorted(ret, key=lambda y: y.num)
# Main thread loop starting the different tasks
def run(self):
@ -141,11 +145,21 @@ class WorkerThread(threading.Thread):
# sometimes tasks (like Upload) don't actually have work to do and are created as already finished
if item.task.stat is STAT_WAITING:
# CalibreTask.start() should wrap all exceptions in it's own error handling
# CalibreTask.start() should wrap all exceptions in its own error handling
item.task.start(self)
# remove self_cleanup tasks and hidden "System Tasks" from list
if item.task.self_cleanup or item.hidden:
self.dequeued.remove(item)
self.queue.task_done()
def end_task(self, task_id):
ins = self.get_instance()
for __, __, __, task, __ in ins.tasks:
if str(task.id) == str(task_id) and task.is_cancellable:
task.stat = STAT_CANCELLED if task.stat == STAT_WAITING else STAT_ENDED
class CalibreTask:
__metaclass__ = abc.ABCMeta
@ -158,10 +172,12 @@ class CalibreTask:
self.end_time = None
self.message = message
self.id = uuid.uuid4()
self.self_cleanup = False
self._scheduled = False
@abc.abstractmethod
def run(self, worker_thread):
"""Provides the caller some human-readable name for this class"""
"""The main entry-point for this task"""
raise NotImplementedError
@abc.abstractmethod
@ -169,6 +185,11 @@ class CalibreTask:
"""Provides the caller some human-readable name for this class"""
raise NotImplementedError
@abc.abstractmethod
def is_cancellable(self):
"""Does this task gracefully handle being cancelled (STAT_ENDED, STAT_CANCELLED)?"""
raise NotImplementedError
def start(self, *args):
self.start_time = datetime.now()
self.stat = STAT_STARTED
@ -178,7 +199,7 @@ class CalibreTask:
self.run(*args)
except Exception as ex:
self._handleError(str(ex))
log.debug_or_exception(ex)
log.error_or_exception(ex)
self.end_time = datetime.now()
@ -219,15 +240,23 @@ class CalibreTask:
We have a separate dictating this because there may be certain tasks that want to override this
"""
# By default, we're good to clean a task if it's "Done"
return self.stat in (STAT_FINISH_SUCCESS, STAT_FAIL)
return self.stat in (STAT_FINISH_SUCCESS, STAT_FAIL, STAT_ENDED, STAT_CANCELLED)
'''@progress.setter
def progress(self, x):
if x > 1:
x = 1
if x < 0:
x = 0
self._progress = x'''
@property
def self_cleanup(self):
return self._self_cleanup
@self_cleanup.setter
def self_cleanup(self, is_self_cleanup):
self._self_cleanup = is_self_cleanup
@property
def scheduled(self):
return self._scheduled
@scheduled.setter
def scheduled(self, is_scheduled):
self._scheduled = is_scheduled
def _handleError(self, error_message):
self.stat = STAT_FAIL

313
cps/shelf.py
View File

@ -23,7 +23,7 @@
import sys
from datetime import datetime
from flask import Blueprint, flash, redirect, request, url_for
from flask import Blueprint, flash, redirect, request, url_for, abort
from flask_babel import gettext as _
from flask_login import current_user, login_required
from sqlalchemy.exc import InvalidRequestError, OperationalError
@ -33,30 +33,12 @@ from . import calibre_db, config, db, logger, ub
from .render_template import render_title_template
from .usermanagement import login_required_if_no_ano
shelf = Blueprint('shelf', __name__)
log = logger.create()
def check_shelf_edit_permissions(cur_shelf):
if not cur_shelf.is_public and not cur_shelf.user_id == int(current_user.id):
log.error("User %s not allowed to edit shelf %s", current_user, cur_shelf)
return False
if cur_shelf.is_public and not current_user.role_edit_shelfs():
log.info("User %s not allowed to edit public shelves", current_user)
return False
return True
shelf = Blueprint('shelf', __name__)
def check_shelf_view_permissions(cur_shelf):
if cur_shelf.is_public:
return True
if current_user.is_anonymous or cur_shelf.user_id != current_user.id:
log.error("User is unauthorized to view non-public shelf: %s", cur_shelf)
return False
return True
@shelf.route("/shelf/add/<int:shelf_id>/<int:book_id>")
@shelf.route("/shelf/add/<int:shelf_id>/<int:book_id>", methods=["POST"])
@login_required
def add_to_shelf(shelf_id, book_id):
xhr = request.headers.get('X-Requested-With') == 'XMLHttpRequest'
@ -64,13 +46,13 @@ def add_to_shelf(shelf_id, book_id):
if shelf is None:
log.error("Invalid shelf specified: %s", shelf_id)
if not xhr:
flash(_(u"Invalid shelf specified"), category="error")
flash(_("Invalid shelf specified"), category="error")
return redirect(url_for('web.index'))
return "Invalid shelf specified", 400
if not check_shelf_edit_permissions(shelf):
if not xhr:
flash(_(u"Sorry you are not allowed to add a book to that shelf"), category="error")
flash(_("Sorry you are not allowed to add a book to that shelf"), category="error")
return redirect(url_for('web.index'))
return "Sorry you are not allowed to add a book to the that shelf", 403
@ -79,7 +61,7 @@ def add_to_shelf(shelf_id, book_id):
if book_in_shelf:
log.error("Book %s is already part of %s", book_id, shelf)
if not xhr:
flash(_(u"Book is already part of the shelf: %(shelfname)s", shelfname=shelf.name), category="error")
flash(_("Book is already part of the shelf: %(shelfname)s", shelfname=shelf.name), category="error")
return redirect(url_for('web.index'))
return "Book is already part of the shelf: %s" % shelf.name, 400
@ -94,17 +76,17 @@ def add_to_shelf(shelf_id, book_id):
try:
ub.session.merge(shelf)
ub.session.commit()
except (OperationalError, InvalidRequestError):
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error("Settings DB is not Writeable")
flash(_(u"Settings DB is not Writeable"), category="error")
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"])
else:
return redirect(url_for('web.index'))
if not xhr:
log.debug("Book has been added to shelf: {}".format(shelf.name))
flash(_(u"Book has been added to shelf: %(sname)s", sname=shelf.name), category="success")
flash(_("Book has been added to shelf: %(sname)s", sname=shelf.name), category="success")
if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"])
else:
@ -112,18 +94,18 @@ def add_to_shelf(shelf_id, book_id):
return "", 204
@shelf.route("/shelf/massadd/<int:shelf_id>")
@shelf.route("/shelf/massadd/<int:shelf_id>", methods=["POST"])
@login_required
def search_to_shelf(shelf_id):
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
if shelf is None:
log.error("Invalid shelf specified: %s", shelf_id)
flash(_(u"Invalid shelf specified"), category="error")
log.error("Invalid shelf specified: {}".format(shelf_id))
flash(_("Invalid shelf specified"), category="error")
return redirect(url_for('web.index'))
if not check_shelf_edit_permissions(shelf):
log.warning("You are not allowed to add a book to the shelf".format(shelf.name))
flash(_(u"You are not allowed to add a book to the shelf"), category="error")
flash(_("You are not allowed to add a book to the shelf"), category="error")
return redirect(url_for('web.index'))
if current_user.id in ub.searched_ids and ub.searched_ids[current_user.id]:
@ -141,7 +123,7 @@ def search_to_shelf(shelf_id):
if not books_for_shelf:
log.error("Books are already part of {}".format(shelf.name))
flash(_(u"Books are already part of the shelf: %(name)s", name=shelf.name), category="error")
flash(_("Books are already part of the shelf: %(name)s", name=shelf.name), category="error")
return redirect(url_for('web.index'))
maxOrder = ub.session.query(func.max(ub.BookShelf.order)).filter(ub.BookShelf.shelf == shelf_id).first()[0] or 0
@ -153,18 +135,18 @@ def search_to_shelf(shelf_id):
try:
ub.session.merge(shelf)
ub.session.commit()
flash(_(u"Books have been added to shelf: %(sname)s", sname=shelf.name), category="success")
except (OperationalError, InvalidRequestError):
flash(_("Books have been added to shelf: %(sname)s", sname=shelf.name), category="success")
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
else:
log.error("Could not add books to shelf: {}".format(shelf.name))
flash(_(u"Could not add books to shelf: %(sname)s", sname=shelf.name), category="error")
flash(_("Could not add books to shelf: %(sname)s", sname=shelf.name), category="error")
return redirect(url_for('web.index'))
@shelf.route("/shelf/remove/<int:shelf_id>/<int:book_id>")
@shelf.route("/shelf/remove/<int:shelf_id>/<int:book_id>", methods=["POST"])
@login_required
def remove_from_shelf(shelf_id, book_id):
xhr = request.headers.get('X-Requested-With') == 'XMLHttpRequest'
@ -197,16 +179,16 @@ def remove_from_shelf(shelf_id, book_id):
ub.session.delete(book_shelf)
shelf.last_modified = datetime.utcnow()
ub.session.commit()
except (OperationalError, InvalidRequestError):
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"])
else:
return redirect(url_for('web.index'))
if not xhr:
flash(_(u"Book has been removed from shelf: %(sname)s", sname=shelf.name), category="success")
flash(_("Book has been removed from shelf: %(sname)s", sname=shelf.name), category="success")
if "HTTP_REFERER" in request.environ:
return redirect(request.environ["HTTP_REFERER"])
else:
@ -215,7 +197,7 @@ def remove_from_shelf(shelf_id, book_id):
else:
if not xhr:
log.warning("You are not allowed to remove a book from shelf: {}".format(shelf.name))
flash(_(u"Sorry you are not allowed to remove a book from this shelf"),
flash(_("Sorry you are not allowed to remove a book from this shelf"),
category="error")
return redirect(url_for('web.index'))
return "Sorry you are not allowed to remove a book from this shelf", 403
@ -224,13 +206,8 @@ def remove_from_shelf(shelf_id, book_id):
@shelf.route("/shelf/create", methods=["GET", "POST"])
@login_required
def create_shelf():
if not current_user.role_edit_shelfs() and request.method == 'POST':
flash(_(u"Sorry you are not allowed to create a public shelf"), category="error")
return redirect(url_for('web.index'))
else:
shelf = ub.Shelf()
return create_edit_shelf(shelf, page_title=_(u"Create a Shelf"), page="shelfcreate")
return create_edit_shelf(shelf, page_title=_("Create a Shelf"), page="shelfcreate")
@shelf.route("/shelf/edit/<int:shelf_id>", methods=["GET", "POST"])
@ -238,102 +215,24 @@ def create_shelf():
def edit_shelf(shelf_id):
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
if not check_shelf_edit_permissions(shelf):
flash(_(u"Sorry you are not allowed to edit this shelf"), category="error")
flash(_("Sorry you are not allowed to edit this shelf"), category="error")
return redirect(url_for('web.index'))
return create_edit_shelf(shelf, page_title=_(u"Edit a shelf"), page="shelfedit", shelf_id=shelf_id)
return create_edit_shelf(shelf, page_title=_("Edit a shelf"), page="shelfedit", shelf_id=shelf_id)
# if shelf ID is set, we are editing a shelf
def create_edit_shelf(shelf, page_title, page, shelf_id=False):
sync_only_selected_shelves = current_user.kobo_only_shelves_sync
# calibre_db.session.query(ub.Shelf).filter(ub.Shelf.user_id == current_user.id).filter(ub.Shelf.kobo_sync).count()
if request.method == "POST":
to_save = request.form.to_dict()
shelf.is_public = 1 if to_save.get("is_public") else 0
if config.config_kobo_sync:
shelf.kobo_sync = True if to_save.get("kobo_sync") else False
shelf_title = to_save.get("title", "")
if check_shelf_is_unique(shelf, shelf_title, shelf_id):
shelf.name = shelf_title
if not shelf_id:
shelf.user_id = int(current_user.id)
ub.session.add(shelf)
shelf_action = "created"
flash_text = _(u"Shelf %(title)s created", title=shelf_title)
else:
shelf_action = "changed"
flash_text = _(u"Shelf %(title)s changed", title=shelf_title)
try:
ub.session.commit()
log.info(u"Shelf {} {}".format(shelf_title, shelf_action))
flash(flash_text, category="success")
return redirect(url_for('shelf.show_shelf', shelf_id=shelf.id))
except (OperationalError, InvalidRequestError) as ex:
ub.session.rollback()
log.debug_or_exception(ex)
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
except Exception as ex:
ub.session.rollback()
log.debug_or_exception(ex)
flash(_(u"There was an error"), category="error")
return render_title_template('shelf_edit.html',
shelf=shelf,
title=page_title,
page=page,
kobo_sync_enabled=config.config_kobo_sync,
sync_only_selected_shelves=sync_only_selected_shelves)
def check_shelf_is_unique(shelf, title, shelf_id=False):
if shelf_id:
ident = ub.Shelf.id != shelf_id
else:
ident = true()
if shelf.is_public == 1:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 1)) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A public shelf with the name '{}' already exists.".format(title))
flash(_(u"A public shelf with the name '%(title)s' already exists.", title=title),
category="error")
else:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 0) &
(ub.Shelf.user_id == int(current_user.id))) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A private shelf with the name '{}' already exists.".format(title))
flash(_(u"A private shelf with the name '%(title)s' already exists.", title=title),
category="error")
return is_shelf_name_unique
def delete_shelf_helper(cur_shelf):
if not cur_shelf or not check_shelf_edit_permissions(cur_shelf):
return
shelf_id = cur_shelf.id
ub.session.delete(cur_shelf)
ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).delete()
ub.session.add(ub.ShelfArchive(uuid=cur_shelf.uuid, user_id=cur_shelf.user_id))
ub.session_commit("successfully deleted Shelf {}".format(cur_shelf.name))
@shelf.route("/shelf/delete/<int:shelf_id>")
@shelf.route("/shelf/delete/<int:shelf_id>", methods=["POST"])
@login_required
def delete_shelf(shelf_id):
cur_shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
try:
delete_shelf_helper(cur_shelf)
except InvalidRequestError:
if not delete_shelf_helper(cur_shelf):
flash(_("Error deleting Shelf"), category="error")
else:
flash(_("Shelf successfully deleted"), category="success")
except InvalidRequestError as e:
ub.session.rollback()
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
return redirect(url_for('web.index'))
@ -354,6 +253,8 @@ def show_shelf(shelf_id, sort_param, page):
@shelf.route("/shelf/order/<int:shelf_id>", methods=["GET", "POST"])
@login_required
def order_shelf(shelf_id):
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
if shelf and check_shelf_view_permissions(shelf):
if request.method == "POST":
to_save = request.form.to_dict()
books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).order_by(
@ -362,24 +263,134 @@ def order_shelf(shelf_id):
for book in books_in_shelf:
setattr(book, 'order', to_save[str(book.book_id)])
counter += 1
# if order diffrent from before -> shelf.last_modified = datetime.utcnow()
# if order different from before -> shelf.last_modified = datetime.utcnow()
try:
ub.session.commit()
except (OperationalError, InvalidRequestError):
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
result = list()
if shelf and check_shelf_view_permissions(shelf):
if shelf:
result = calibre_db.session.query(db.Books) \
.join(ub.BookShelf, ub.BookShelf.book_id == db.Books.id, isouter=True) \
.add_columns(calibre_db.common_filters().label("visible")) \
.filter(ub.BookShelf.shelf == shelf_id).order_by(ub.BookShelf.order.asc()).all()
return render_title_template('shelf_order.html', entries=result,
title=_(u"Change order of Shelf: '%(name)s'", name=shelf.name),
title=_("Change order of Shelf: '%(name)s'", name=shelf.name),
shelf=shelf, page="shelforder")
else:
abort(404)
def check_shelf_edit_permissions(cur_shelf):
if not cur_shelf.is_public and not cur_shelf.user_id == int(current_user.id):
log.error("User {} not allowed to edit shelf: {}".format(current_user.id, cur_shelf.name))
return False
if cur_shelf.is_public and not current_user.role_edit_shelfs():
log.info("User {} not allowed to edit public shelves".format(current_user.id))
return False
return True
def check_shelf_view_permissions(cur_shelf):
if cur_shelf.is_public:
return True
if current_user.is_anonymous or cur_shelf.user_id != current_user.id:
log.error("User is unauthorized to view non-public shelf: {}".format(cur_shelf.name))
return False
return True
# if shelf ID is set, we are editing a shelf
def create_edit_shelf(shelf, page_title, page, shelf_id=False):
sync_only_selected_shelves = current_user.kobo_only_shelves_sync
# calibre_db.session.query(ub.Shelf).filter(ub.Shelf.user_id == current_user.id).filter(ub.Shelf.kobo_sync).count()
if request.method == "POST":
to_save = request.form.to_dict()
if not current_user.role_edit_shelfs() and to_save.get("is_public") == "on":
flash(_("Sorry you are not allowed to create a public shelf"), category="error")
return redirect(url_for('web.index'))
is_public = 1 if to_save.get("is_public") == "on" else 0
if config.config_kobo_sync:
shelf.kobo_sync = True if to_save.get("kobo_sync") else False
if shelf.kobo_sync:
ub.session.query(ub.ShelfArchive).filter(ub.ShelfArchive.user_id == current_user.id).filter(
ub.ShelfArchive.uuid == shelf.uuid).delete()
ub.session_commit()
shelf_title = to_save.get("title", "")
if check_shelf_is_unique(shelf_title, is_public, shelf_id):
shelf.name = shelf_title
shelf.is_public = is_public
if not shelf_id:
shelf.user_id = int(current_user.id)
ub.session.add(shelf)
shelf_action = "created"
flash_text = _("Shelf %(title)s created", title=shelf_title)
else:
shelf_action = "changed"
flash_text = _("Shelf %(title)s changed", title=shelf_title)
try:
ub.session.commit()
log.info("Shelf {} {}".format(shelf_title, shelf_action))
flash(flash_text, category="success")
return redirect(url_for('shelf.show_shelf', shelf_id=shelf.id))
except (OperationalError, InvalidRequestError) as ex:
ub.session.rollback()
log.error_or_exception(ex)
log.error_or_exception("Settings Database error: {}".format(ex))
flash(_("Oops! Database Error: %(error)s.", error=ex.orig), category="error")
except Exception as ex:
ub.session.rollback()
log.error_or_exception(ex)
flash(_("There was an error"), category="error")
return render_title_template('shelf_edit.html',
shelf=shelf,
title=page_title,
page=page,
kobo_sync_enabled=config.config_kobo_sync,
sync_only_selected_shelves=sync_only_selected_shelves)
def check_shelf_is_unique(title, is_public, shelf_id=False):
if shelf_id:
ident = ub.Shelf.id != shelf_id
else:
ident = true()
if is_public == 1:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 1)) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A public shelf with the name '{}' already exists.".format(title))
flash(_("A public shelf with the name '%(title)s' already exists.", title=title),
category="error")
else:
is_shelf_name_unique = ub.session.query(ub.Shelf) \
.filter((ub.Shelf.name == title) & (ub.Shelf.is_public == 0) &
(ub.Shelf.user_id == int(current_user.id))) \
.filter(ident) \
.first() is None
if not is_shelf_name_unique:
log.error("A private shelf with the name '{}' already exists.".format(title))
flash(_("A private shelf with the name '%(title)s' already exists.", title=title),
category="error")
return is_shelf_name_unique
def delete_shelf_helper(cur_shelf):
if not cur_shelf or not check_shelf_edit_permissions(cur_shelf):
return False
shelf_id = cur_shelf.id
ub.session.delete(cur_shelf)
ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).delete()
ub.session.add(ub.ShelfArchive(uuid=cur_shelf.uuid, user_id=cur_shelf.user_id))
ub.session_commit("successfully deleted Shelf {}".format(cur_shelf.name))
return True
def change_shelf_order(shelf_id, order):
@ -399,7 +410,6 @@ def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
# check user is allowed to access shelf
if shelf and check_shelf_view_permissions(shelf):
if shelf_type == 1:
# order = [ub.BookShelf.order.asc()]
if sort_param == 'pubnew':
@ -430,6 +440,7 @@ def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
db.Books,
ub.BookShelf.shelf == shelf_id,
[ub.BookShelf.order.asc()],
True, config.config_read_column,
ub.BookShelf, ub.BookShelf.book_id == db.Books.id)
# delete chelf entries where book is not existent anymore, can happen if book is deleted outside calibre-web
wrong_entries = calibre_db.session.query(ub.BookShelf) \
@ -440,17 +451,17 @@ def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
try:
ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == entry.book_id).delete()
ub.session.commit()
except (OperationalError, InvalidRequestError):
except (OperationalError, InvalidRequestError) as e:
ub.session.rollback()
log.error("Settings DB is not Writeable")
flash(_("Settings DB is not Writeable"), category="error")
log.error_or_exception("Settings Database error: {}".format(e))
flash(_("Oops! Database Error: %(error)s.", error=e.orig), category="error")
return render_title_template(page,
entries=result,
pagination=pagination,
title=_(u"Shelf: '%(name)s'", name=shelf.name),
title=_("Shelf: '%(name)s'", name=shelf.name),
shelf=shelf,
page="shelf")
else:
flash(_(u"Error opening shelf. Shelf does not exist or is not accessible"), category="error")
flash(_("Error opening shelf. Shelf does not exist or is not accessible"), category="error")
return redirect(url_for("web.index"))

101
cps/static/css/caliBlur.css
View File

@ -3290,9 +3290,11 @@ div.btn-group[role=group][aria-label="Download, send to Kindle, reading"] .dropd
-ms-transform-origin: center top;
transform-origin: center top;
border: 0;
left: 0 !important;
overflow-y: auto;
}
.dropdown-menu:not(.datepicker-dropdown):not(.profileDropli) {
left: 0 !important;
}
#add-to-shelves {
max-height: calc(100% - 120px);
overflow-y: auto;
@ -4423,38 +4425,6 @@ body.advanced_search > div.container-fluid > div.row-fluid > div.col-sm-10 > div
left: 49px;
margin-top: 5px
}
body:not(.blur) > .navbar > .container-fluid > .navbar-header:after, body:not(.blur) > .navbar > .container-fluid > .navbar-header:before {
color: hsla(0, 0%, 100%, .7);
cursor: pointer;
display: block;
font-family: plex-icons-new, serif;
font-size: 20px;
font-stretch: 100%;
font-style: normal;
font-variant-caps: normal;
font-variant-east-asian: normal;
font-variant-numeric: normal;
font-weight: 400;
height: 60px;
letter-spacing: normal;
line-height: 60px;
position: absolute
}
body:not(.blur) > .navbar > .container-fluid > .navbar-header:before {
content: "\EA30";
-webkit-font-variant-ligatures: normal;
font-variant-ligatures: normal;
left: 20px
}
body:not(.blur) > .navbar > .container-fluid > .navbar-header:after {
content: "\EA2F";
-webkit-font-variant-ligatures: normal;
font-variant-ligatures: normal;
left: 60px
}
}
body.admin > div.container-fluid > div > div.col-sm-10 > div.container-fluid > div.row:first-of-type > div.col > h2:before, body.admin > div.container-fluid > div > div.col-sm-10 > div.discover > h2:first-of-type:before, body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.discover > h1:before, body.newuser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.discover > h1:before {
@ -5150,7 +5120,7 @@ body.login > div.navbar.navbar-default.navbar-static-top > div > div.navbar-head
pointer-events: none
}
#DeleteDomain:hover:before, #RestartDialog:hover:before, #ShutdownDialog:hover:before, #StatusDialog:hover:before, #deleteButton, #deleteModal:hover:before, body.mailset > div.container-fluid > div > div.col-sm-10 > div.discover td > a:hover {
#DeleteDomain:hover:before, #RestartDialog:hover:before, #ShutdownDialog:hover:before, #StatusDialog:hover:before, #deleteButton, #deleteModal:hover:before, #cancelTaskModal:hover:before, body.mailset > div.container-fluid > div > div.col-sm-10 > div.discover td > a:hover {
cursor: pointer
}
@ -5237,7 +5207,11 @@ body.admin > div.container-fluid > div > div.col-sm-10 > div.container-fluid > d
margin-bottom: 20px
}
body.admin:not(.modal-open) .btn-default {
body.admin > div.container-fluid div.scheduled_tasks_details {
margin-bottom: 20px
}
body.admin .btn-default {
margin-bottom: 10px
}
@ -5468,7 +5442,7 @@ body.admin.modal-open .navbar {
z-index: 0 !important
}
#RestartDialog, #ShutdownDialog, #StatusDialog, #deleteModal {
#RestartDialog, #ShutdownDialog, #StatusDialog, #deleteModal, #cancelTaskModal {
top: 0;
overflow: hidden;
padding-top: 70px;
@ -5478,7 +5452,7 @@ body.admin.modal-open .navbar {
background: rgba(0, 0, 0, .5)
}
#RestartDialog:before, #ShutdownDialog:before, #StatusDialog:before, #deleteModal:before {
#RestartDialog:before, #ShutdownDialog:before, #StatusDialog:before, #deleteModal:before, #cancelTaskModal:before {
content: "\E208";
padding-right: 10px;
display: block;
@ -5500,18 +5474,18 @@ body.admin.modal-open .navbar {
z-index: 99
}
#RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before {
#RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before, #cancelTaskModal.in:before {
-webkit-transform: translate(0, 0);
-ms-transform: translate(0, 0);
transform: translate(0, 0)
}
#RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog {
#RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog, #cancelTaskModal > .modal-dialog {
width: 450px;
margin: auto
}
#RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content {
#RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content, #cancelTaskModal > .modal-dialog > .modal-content {
max-height: calc(100% - 90px);
-webkit-box-shadow: 0 5px 15px rgba(0, 0, 0, .5);
box-shadow: 0 5px 15px rgba(0, 0, 0, .5);
@ -5522,7 +5496,7 @@ body.admin.modal-open .navbar {
width: 450px
}
#RestartDialog > .modal-dialog > .modal-content > .modal-header, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header, #StatusDialog > .modal-dialog > .modal-content > .modal-header, #deleteModal > .modal-dialog > .modal-content > .modal-header {
#RestartDialog > .modal-dialog > .modal-content > .modal-header, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header, #StatusDialog > .modal-dialog > .modal-content > .modal-header, #deleteModal > .modal-dialog > .modal-content > .modal-header, #cancelTaskModal > .modal-dialog > .modal-content > .modal-header {
padding: 15px 20px;
border-radius: 3px 3px 0 0;
line-height: 1.71428571;
@ -5535,7 +5509,7 @@ body.admin.modal-open .navbar {
text-align: left
}
#RestartDialog > .modal-dialog > .modal-content > .modal-header:before, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header:before, #StatusDialog > .modal-dialog > .modal-content > .modal-header:before, #deleteModal > .modal-dialog > .modal-content > .modal-header:before {
#RestartDialog > .modal-dialog > .modal-content > .modal-header:before, #ShutdownDialog > .modal-dialog > .modal-content > .modal-header:before, #StatusDialog > .modal-dialog > .modal-content > .modal-header:before, #deleteModal > .modal-dialog > .modal-content > .modal-header:before, #cancelTaskModal > .modal-dialog > .modal-content > .modal-header:before {
padding-right: 10px;
font-size: 18px;
color: #999;
@ -5564,6 +5538,11 @@ body.admin.modal-open .navbar {
font-family: plex-icons-new, serif
}
#cancelTaskModal > .modal-dialog > .modal-content > .modal-header:before {
content: "\EA6D";
font-family: plex-icons-new, serif
}
#RestartDialog > .modal-dialog > .modal-content > .modal-header:after {
content: "Restart Calibre-Web";
display: inline-block;
@ -5588,7 +5567,13 @@ body.admin.modal-open .navbar {
font-size: 20px
}
#StatusDialog > .modal-dialog > .modal-content > .modal-header > span, #deleteModal > .modal-dialog > .modal-content > .modal-header > span, #loader > center > img, .rating-mobile {
#cancelTaskModal > .modal-dialog > .modal-content > .modal-header:after {
content: "Delete Book";
display: inline-block;
font-size: 20px
}
#StatusDialog > .modal-dialog > .modal-content > .modal-header > span, #deleteModal > .modal-dialog > .modal-content > .modal-header > span, #cancelTaskModal > .modal-dialog > .modal-content > .modal-header > span, #loader > center > img, .rating-mobile {
display: none
}
@ -5602,7 +5587,7 @@ body.admin.modal-open .navbar {
text-align: left
}
#ShutdownDialog > .modal-dialog > .modal-content > .modal-body, #StatusDialog > .modal-dialog > .modal-content > .modal-body, #deleteModal > .modal-dialog > .modal-content > .modal-body {
#ShutdownDialog > .modal-dialog > .modal-content > .modal-body, #StatusDialog > .modal-dialog > .modal-content > .modal-body, #deleteModal > .modal-dialog > .modal-content > .modal-body, #cancelTaskModal > .modal-dialog > .modal-content > .modal-body {
padding: 20px 20px 40px;
font-size: 16px;
line-height: 1.6em;
@ -5612,7 +5597,7 @@ body.admin.modal-open .navbar {
text-align: left
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > p, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > p, #StatusDialog > .modal-dialog > .modal-content > .modal-body > p, #deleteModal > .modal-dialog > .modal-content > .modal-body > p {
#RestartDialog > .modal-dialog > .modal-content > .modal-body > p, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > p, #StatusDialog > .modal-dialog > .modal-content > .modal-body > p, #deleteModal > .modal-dialog > .modal-content > .modal-body > p, #cancelTaskModal > .modal-dialog > .modal-content > .modal-body > p {
padding: 20px 20px 0 0;
font-size: 16px;
line-height: 1.6em;
@ -5621,7 +5606,7 @@ body.admin.modal-open .navbar {
background: #282828
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart), #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown), #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default {
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart), #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown), #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default, #cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-default {
float: right;
z-index: 9;
position: relative;
@ -5669,6 +5654,18 @@ body.admin.modal-open .navbar {
border-radius: 3px
}
#cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-danger {
float: right;
z-index: 9;
position: relative;
margin: 0 0 0 10px;
min-width: 80px;
padding: 10px 18px;
font-size: 16px;
line-height: 1.33;
border-radius: 3px
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart) {
margin: 25px 0 0 10px
}
@ -5681,7 +5678,11 @@ body.admin.modal-open .navbar {
margin: 0 0 0 10px
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart):hover, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown):hover, #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default:hover {
#cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-default {
margin: 0 0 0 10px
}
#RestartDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#restart):hover, #ShutdownDialog > .modal-dialog > .modal-content > .modal-body > .btn-default:not(#shutdown):hover, #deleteModal > .modal-dialog > .modal-content > .modal-footer > .btn-default:hover, #cancelTaskModal > .modal-dialog > .modal-content > .modal-footer > .btn-default:hover {
background-color: hsla(0, 0%, 100%, .3)
}
@ -7303,11 +7304,11 @@ body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.
background-color: transparent !important
}
#RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog {
#RestartDialog > .modal-dialog, #ShutdownDialog > .modal-dialog, #StatusDialog > .modal-dialog, #deleteModal > .modal-dialog, #cancelTaskModal > .modal-dialog {
max-width: calc(100vw - 40px)
}
#RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content {
#RestartDialog > .modal-dialog > .modal-content, #ShutdownDialog > .modal-dialog > .modal-content, #StatusDialog > .modal-dialog > .modal-content, #deleteModal > .modal-dialog > .modal-content, #cancelTaskModal > .modal-dialog > .modal-content {
max-width: calc(100vw - 40px);
left: 0
}
@ -7457,7 +7458,7 @@ body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.
padding: 30px 15px
}
#RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before {
#RestartDialog.in:before, #ShutdownDialog.in:before, #StatusDialog.in:before, #deleteModal.in:before, #cancelTaskModal.in:before {
left: auto;
right: 34px
}

4
cps/static/css/caliBlur_override.css
View File

@ -22,3 +22,7 @@ body.serieslist.grid-view div.container-fluid > div > div.col-sm-10::before {
padding: 0 0;
line-height: 15px;
}
input.datepicker {color: transparent}
input.datepicker:focus {color: transparent}
input.datepicker:focus + input {color: #555}

19
cps/static/css/epub_themes.css Normal file
View File

@ -0,0 +1,19 @@
.lightTheme {
background: #fff;
color: #000;
}
.darkTheme {
background: #202124;
color: #fff
}
.sepiaTheme {
background: #ece1ca;
color: #000;
}
.blackTheme {
background: #000;
color: #fff
}

6
cps/static/css/libs/images/findbarButton-next-dark.svg
View File

@ -1,6 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M8 12a1 1 0 0 1-.707-.293l-5-5a1 1 0 0 1 1.414-1.414L8
9.586l4.293-4.293a1 1 0 0 1 1.414 1.414l-5 5A1 1 0 0 1 8 12z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 461 B

5
cps/static/css/libs/images/findbarButton-previous-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M13 11a1 1 0 0 1-.707-.293L8 6.414l-4.293 4.293a1 1 0 0 1-1.414-1.414l5-5a1 1 0 0 1 1.414 0l5 5A1 1 0 0 1 13 11z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 458 B

BIN
cps/static/css/libs/images/grab.cur
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 326 B

BIN
cps/static/css/libs/images/grabbing.cur
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 326 B

16
cps/static/css/libs/images/secondaryToolbarButton-documentProperties-dark.svg
View File

@ -1,16 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16
16"
fill="rgba(255,255,255,1)">
<path
d="M8 16a8 8 0 1 1 8-8 8.009 8.009 0 0 1-8 8zM8 2a6 6 0 1 0 6 6 6.006 6.006 0 0 0-6-6z">
</path>
<path
d="M8 7a1 1 0 0 0-1 1v3a1 1 0 0 0 2 0V8a1 1 0 0 0-1-1z">
</path>
<circle
cx="8" cy="5" r="1.188">
</circle>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 557 B

2
cps/static/css/libs/images/secondaryToolbarButton-firstPage-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M13 13c-.3 0-.5-.1-.7-.3L8 8.4l-4.3 4.3c-.9.9-2.3-.5-1.4-1.4l5-5c.4-.4 1-.4 1.4 0l5 5c.6.6.2 1.7-.7 1.7zm0-11H3C1.7 2 1.7 4 3 4h10c1.3 0 1.3-2 0-2z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 255 B

2
cps/static/css/libs/images/secondaryToolbarButton-handTool-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M15 3.7V13c0 1.5-1.53 3-3 3H7.13c-.72 0-1.63-.5-2.13-1l-5-5s.84-1 .87-1c.13-.1.33-.2.53-.2.1 0 .3.1.4.2L4 10.6V2.7c0-.6.4-1 1-1s1 .4 1 1v4.6h1V1c0-.6.4-1 1-1s1 .4 1 1v6.3h1V1.7c0-.6.4-1 1-1s1 .4 1 1v5.7h1V3.7c0-.6.4-1 1-1s1 .4 1 1z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 339 B

2
cps/static/css/libs/images/secondaryToolbarButton-lastPage-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M8 10c-.3 0-.5-.1-.7-.3l-5-5c-.9-.9.5-2.3 1.4-1.4L8 7.6l4.3-4.3c.9-.9 2.3.5 1.4 1.4l-5 5c-.2.2-.4.3-.7.3zm5 2H3c-1.3 0-1.3 2 0 2h10c1.3 0 1.3-2 0-2z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 256 B

2
cps/static/css/libs/images/secondaryToolbarButton-rotateCcw-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M1 1a1 1 0 011 1v2.4A7 7 0 118 15a7 7 0 01-4.9-2 1 1 0 011.4-1.5 5 5 0 10-1-5.5H6a1 1 0 010 2H1a1 1 0 01-1-1V2a1 1 0 011-1z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 231 B

5
cps/static/css/libs/images/secondaryToolbarButton-rotateCw-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M15 1a1 1 0 0 0-1 1v2.418A6.995 6.995 0 1 0 8 15a6.954 6.954 0 0 0 4.95-2.05 1 1 0 0 0-1.414-1.414A5.019 5.019 0 1 1 12.549 6H10a1 1 0 0 0 0 2h5a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 521 B

2
cps/static/css/libs/images/secondaryToolbarButton-scrollHorizontal-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M0 4h1.5c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5H0zM9.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C5 4.5 5.5 4 6.5 4zM16 4h-1.5c-1 0-1.5.5-1.5 1.5v5c0 1 .5 1.5 1.5 1.5H16z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 302 B

1
cps/static/css/libs/images/secondaryToolbarButton-scrollPage.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"><path d="M9.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C5 4.5 5.5 4 6.5 4z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 171 B

2
cps/static/css/libs/images/secondaryToolbarButton-scrollVertical-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M9.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C5 4.5 5.5 4 6.5 4zM11 0v.5c0 1-.5 1.5-1.5 1.5h-3C5.5 2 5 1.5 5 .5V0h6zM11 16v-.5c0-1-.5-1.5-1.5-1.5h-3c-1 0-1.5.5-1.5 1.5v.5h6z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 307 B

2
cps/static/css/libs/images/secondaryToolbarButton-scrollWrapped-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M5.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5C1 4.5 1.5 4 2.5 4zM7 0v.5C7 1.5 6.5 2 5.5 2h-3C1.5 2 1 1.5 1 .5V0h6zM7 16v-.5c0-1-.5-1.5-1.5-1.5h-3c-1 0-1.5.5-1.5 1.5v.5h6zM13.5 4c1 0 1.5.5 1.5 1.5v5c0 1-.5 1.5-1.5 1.5h-3c-1 0-1.5-.5-1.5-1.5v-5c0-1 .5-1.5 1.5-1.5zM15 0v.5c0 1-.5 1.5-1.5 1.5h-3C9.5 2 9 1.5 9 .5V0h6zM15 16v-.507c0-1-.5-1.5-1.5-1.5h-3C9.5 14 9 14.5 9 15.5v.5h6z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 509 B

5
cps/static/css/libs/images/secondaryToolbarButton-selectTool-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M12.408 8.217l-8.083-6.7A.2.2 0 0 0 4 1.672V12.3a.2.2 0 0 0 .333.146l2.56-2.372 1.857 3.9A1.125 1.125 0 1 0 10.782 13L8.913 9.075l3.4-.51a.2.2 0 0 0 .095-.348z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 505 B

2
cps/static/css/libs/images/secondaryToolbarButton-spreadEven-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M1.5 3.5C.5 3.5 0 4 0 5v6.5c0 1 .5 1.5 1.5 1.5h4c1 0 1.5-.5 1.5-1.5V5c0-1-.5-1.5-1.5-1.5zm2 1.2c.8 0 1.4.2 1.8.6.5.4.7 1 .7 1.7 0 .5-.2 1-.5 1.4-.2.3-.5.7-1 1l-.6.4c-.4.3-.6.4-.75.56-.15.14-.25.24-.35.44H6v1.3H1c0-.6.1-1.1.3-1.5.3-.6.7-1 1.5-1.6.7-.4 1.1-.8 1.28-1 .32-.3.42-.6.42-1 0-.3-.1-.6-.23-.8-.17-.2-.37-.3-.77-.3s-.7.1-.9.5c-.04.2-.1.5-.1.9H1.1c0-.6.1-1.1.3-1.5.4-.7 1.1-1.1 2.1-1.1zM10.54 3.54C9.5 3.54 9 4 9 5v6.5c0 1 .5 1.5 1.54 1.5h4c.96 0 1.46-.5 1.46-1.5V5c0-1-.5-1.46-1.5-1.46zm1.9.95c.7 0 1.3.2 1.7.5.4.4.6.8.6 1.4 0 .4-.1.8-.4 1.1-.2.2-.3.3-.5.4.1 0 .3.1.6.3.4.3.5.8.5 1.4 0 .6-.2 1.2-.6 1.6-.4.5-1.1.7-1.9.7-1 0-1.8-.3-2.2-1-.14-.29-.24-.69-.24-1.29h1.4c0 .3 0 .5.1.7.2.4.5.5 1 .5.3 0 .5-.1.7-.3.2-.2.3-.5.3-.8 0-.5-.2-.8-.6-.95-.2-.05-.5-.15-1-.15v-1c.5 0 .8-.1 1-.14.3-.1.5-.4.5-.9 0-.3-.1-.5-.2-.7-.2-.2-.4-.3-.7-.3-.3 0-.6.1-.75.3-.2.2-.2.5-.2.86h-1.34c0-.4.1-.7.19-1.1 0-.12.2-.32.4-.62.2-.2.4-.3.7-.4.3-.1.6-.1 1-.1z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 1.0 KiB

2
cps/static/css/libs/images/secondaryToolbarButton-spreadNone-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M6 3c-1 0-1.5.5-1.5 1.5v7c0 1 .5 1.5 1.5 1.5h4c1 0 1.5-.5 1.5-1.5v-7c0-1-.5-1.5-1.5-1.5z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 196 B

2
cps/static/css/libs/images/secondaryToolbarButton-spreadOdd-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M10.56 3.5C9.56 3.5 9 4 9 5v6.5c0 1 .5 1.5 1.5 1.5h4c1 0 1.5-.5 1.5-1.5V5c0-1-.5-1.5-1.5-1.5zm1.93 1.2c.8 0 1.4.2 1.8.64.5.4.7 1 .7 1.7 0 .5-.2 1-.5 1.44-.2.3-.6.6-1 .93l-.6.4c-.4.3-.6.4-.7.55-.1.1-.2.2-.3.4h3.2v1.27h-5c0-.5.1-1 .3-1.43.2-.49.7-1 1.5-1.54.7-.5 1.1-.8 1.3-1.02.3-.3.4-.7.4-1.05 0-.3-.1-.6-.3-.77-.2-.2-.4-.3-.7-.3-.4 0-.7.2-.9.5-.1.2-.1.5-.2.9h-1.4c0-.6.2-1.1.3-1.5.4-.7 1.1-1.1 2-1.1zM1.54 3.5C.54 3.5 0 4 0 5v6.5c0 1 .5 1.5 1.54 1.5h4c1 0 1.5-.5 1.5-1.5V5c0-1-.5-1.5-1.5-1.5zm1.8 1.125H4.5V12H3V6.9H1.3v-1c.5 0 .8 0 .97-.03.33-.07.53-.17.73-.37.1-.2.2-.3.25-.5.05-.2.05-.3.05-.3z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 705 B

2
cps/static/css/libs/images/toolbarButton-bookmark-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M4 16V2s0-1 1-1h6s1 0 1 1v14l-4-5z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 142 B

1
cps/static/css/libs/images/toolbarButton-currentOutlineItem.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"><path d="m14 9h-6c-1.3 0-1.3 2 0 2h6c1.3 0 1.3-2 0-2zm-5.2-8h-3.8c-1.3 0-1.3 2 0 2h1.7zm-6.8 0c-1 0-1.3 1-0.7 1.7 0.7 0.6 1.7 0.3 1.7-0.7 0-0.5-0.4-1-1-1zm3 8c-1 0-1.3 1-0.7 1.7 0.6 0.6 1.7 0.2 1.7-0.7 0-0.5-0.4-1-1-1zm0.3-4h-0.3c-1.4 0-1.4 2 0 2h2.3zm-3.3 0c-0.9 0-1.4 1-0.7 1.7 0.7 0.6 1.7 0.2 1.7-0.7 0-0.6-0.5-1-1-1zm12 8h-9c-1.3 0-1.3 2 0 2h9c1.3 0 1.3-2 0-2zm-12 0c-1 0-1.3 1-0.7 1.7 0.7 0.6 1.7 0.2 1.7-0.712 0-0.5-0.4-1-1-1z"/><path d="m7.37 4.838 3.93-3.911v2.138h3.629v3.546h-3.629v2.138l-3.93-3.911"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 581 B

5
cps/static/css/libs/images/toolbarButton-download-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M14 3h-2v2h2v8H2V5h7V3h-.849L6.584 1.538A2 2 0 0 0 5.219 1H2a2 2 0 0 0-2 2v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V5a2 2 0 0 0-2-2zM2 3h3.219l1.072 1H2z"></path><path d="M8.146 6.146a.5.5 0 0 0 0 .707l2 2a.5.5 0 0 0 .707 0l2-2a.5.5 0 1 0-.707-.707L11 7.293V.5a.5.5 0 0 0-1 0v6.793L8.854 6.146a.5.5 0 0 0-.708 0z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 651 B

24
cps/static/css/libs/images/toolbarButton-editorFreeText.svg Normal file
View File

@ -0,0 +1,24 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- copied from https://www.svgrepo.com/svg/255881/text -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 16 16" style="enable-background:new 0 0 16 16;" xml:space="preserve">
<g>
<g transform="scale(0.03125)">
<path d="M405.787,43.574H8.17c-4.513,0-8.17,3.658-8.17,8.17v119.83c0,4.512,3.657,8.17,8.17,8.17h32.681
c4.513,0,8.17-3.658,8.17-8.17v-24.511h95.319v119.83c0,4.512,3.657,8.17,8.17,8.17c4.513,0,8.17-3.658,8.17-8.17v-128
c0-4.512-3.657-8.17-8.17-8.17H40.851c-4.513,0-8.17,3.658-8.17,8.17v24.511H16.34V59.915h381.277v103.489h-16.34v-24.511
c0-4.512-3.657-8.17-8.17-8.17h-111.66c-4.513,0-8.17,3.658-8.17,8.17v288.681c0,4.512,3.657,8.17,8.17,8.17h57.191v16.34H95.319
v-16.34h57.191c4.513,0,8.17-3.658,8.17-8.17v-128c0-4.512-3.657-8.17-8.17-8.17c-4.513,0-8.17,3.658-8.17,8.17v119.83H87.149
c-4.513,0-8.17,3.658-8.17,8.17v32.681c0,4.512,3.657,8.17,8.17,8.17h239.66c4.513,0,8.17-3.658,8.17-8.17v-32.681
c0-4.512-3.657-8.17-8.17-8.17h-57.192v-272.34h95.319v24.511c0,4.512,3.657,8.17,8.17,8.17h32.681c4.513,0,8.17-3.658,8.17-8.17
V51.745C413.957,47.233,410.3,43.574,405.787,43.574z"/>
</g>
</g>
<g>
<g transform="scale(0.03125)">
<path d="M503.83,452.085h-24.511V59.915h24.511c4.513,0,8.17-3.658,8.17-8.17s-3.657-8.17-8.17-8.17h-65.362
c-4.513,0-8.17,3.658-8.17,8.17s3.657,8.17,8.17,8.17h24.511v392.17h-24.511c-4.513,0-8.17,3.658-8.17,8.17s3.657,8.17,8.17,8.17
h65.362c4.513,0,8.17-3.658,8.17-8.17S508.343,452.085,503.83,452.085z"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.6 KiB

9
cps/static/css/libs/images/toolbarButton-editorInk.svg Normal file
View File

@ -0,0 +1,9 @@
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE svg PUBLIC '-//W3C//DTD SVG 1.1//EN' 'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd'>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16" xmlns:xlink="http://www.w3.org/1999/xlink" enable-background="new 0 0 16 16">
<g>
<g transform="scale(0.03125)">
<path d="m455.1,137.9l-32.4,32.4-81-81.1 32.4-32.4c6.6-6.6 18.1-6.6 24.7,0l56.3,56.4c6.8,6.8 6.8,17.9 0,24.7zm-270.7,271l-81-81.1 209.4-209.7 81,81.1-209.4,209.7zm-99.7-42l60.6,60.7-84.4,23.8 23.8-84.5zm399.3-282.6l-56.3-56.4c-11-11-50.7-31.8-82.4,0l-285.3,285.5c-2.5,2.5-4.3,5.5-5.2,8.9l-43,153.1c-2,7.1 0.1,14.7 5.2,20 5.2,5.3 15.6,6.2 20,5.2l153-43.1c3.4-0.9 6.4-2.7 8.9-5.2l285.1-285.5c22.7-22.7 22.7-59.7 0-82.5z"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 804 B

1
cps/static/css/libs/images/toolbarButton-menuArrow-dark.svg
View File

@ -1 +0,0 @@
<svg width="16" height="16" xmlns="http://www.w3.org/2000/svg" fill="rgba(255,255,255,1)"><path d="M8 11a1 1 0 01-.707-.293l-2.99-2.99c-.91-.942.471-2.324 1.414-1.414L8 8.586l2.283-2.283c.943-.91 2.324.472 1.414 1.414l-2.99 2.99A1 1 0 018 11z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 251 B

5
cps/static/css/libs/images/toolbarButton-openFile-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M14.859 3.2a1.335 1.335 0 0 1-1.217.8H13v1h1v8H2V5h8V4h-.642a1.365 1.365 0 0 1-1.325-1.11L6.584 1.538A2 2 0 0 0 5.219 1H2a2 2 0 0 0-2 2v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V5a2 2 0 0 0-1.141-1.8zM2 3h3.219l1.072 1H2zm7.854-.146L11 1.707V8.5a.5.5 0 0 0 1 0V1.707l1.146 1.146a.5.5 0 1 0 .707-.707l-2-2a.5.5 0 0 0-.707 0l-2 2a.5.5 0 0 0 .707.707z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 686 B

8
cps/static/css/libs/images/toolbarButton-pageDown-dark.svg
View File

@ -1,8 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16
16"
fill="rgba(255,255,255,1)"><path transform='rotate(90) translate(0, -16)'
d="M15.707 7.293l-6-6a1 1 0 0 0-1.414 1.414L12.586 7H1a1 1 0 0 0 0 2h11.586l-4.293
4.293a1 1 0 1 0 1.414 1.414l6-6a1 1 0 0 0 0-1.414z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 517 B

13
cps/static/css/libs/images/toolbarButton-pageUp-dark.svg
View File

@ -1,13 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16
16"
fill="rgba(255,255,255,1)">
<path
transform='rotate(90) translate(0, -16)'
d="M15 7H3.414l4.293-4.293a1 1 0 0
0-1.414-1.414l-6 6a1 1 0 0 0 0 1.414l6 6a1 1 0 0 0 1.414-1.414L3.414 9H15a1 1 0 0
0 0-2z">
</path>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 517 B

2
cps/static/css/libs/images/toolbarButton-presentationMode-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16"
fill="rgba(255,255,255,1)"><path d="M.5 1H7s0-1 1-1 1 1 1 1h6.5s.5 0 .5.5-.5.5-.5.5H.5S0 2 0 1.5.5 1 .5 1zM1 3h14v7c0 2-1 2-2 2H3c-1 0-2 0-2-2zm5 1v7l6-3.5zM3.72 15.33l.53-2s0-.5.65-.35c.51.13.38.63.38.63l-.53 2s0 .5-.64.35c-.53-.13-.39-.63-.39-.63zM11.24 15.61l-.53-1.99s0-.5.38-.63c.51-.13.64.35.64.35l.53 2s0 .5-.38.63c-.5.13-.64-.35-.65-.35z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 417 B

5
cps/static/css/libs/images/toolbarButton-print-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M14 5h-1V1a1 1 0 0 0-1-1H4a1 1 0 0 0-1 1v4H2a2 2 0 0 0-2 2v5h3v3a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1v-3h3V7a2 2 0 0 0-2-2zM2.5 8a.5.5 0 1 1 .5-.5.5.5 0 0 1-.5.5zm9.5 7H4v-5h8zm0-10H4V1h8zm-6.5 7h4a.5.5 0 0 0 0-1h-4a.5.5 0 1 0 0 1zm0 2h5a.5.5 0 0 0 0-1h-5a.5.5 0 1 0 0 1z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 610 B

5
cps/static/css/libs/images/toolbarButton-search-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M15.707 14.293l-4.822-4.822a6.019 6.019 0 1 0-1.414 1.414l4.822 4.822a1 1 0 0 0 1.414-1.414zM6 10a4 4 0 1 1 4-4 4 4 0 0 1-4 4z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 472 B

5
cps/static/css/libs/images/toolbarButton-secondaryToolbarToggle-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M8.707 7.293l-5-5a1 1 0 0 0-1.414 1.414L6.586 8l-4.293 4.293a1 1 0 1 0 1.414 1.414l5-5a1 1 0 0 0 0-1.414zm6 0l-5-5a1 1 0 0 0-1.414 1.414L12.586 8l-4.293 4.293a1 1 0 1 0 1.414 1.414l5-5a1 1 0 0 0 0-1.414z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 549 B

5
cps/static/css/libs/images/toolbarButton-sidebarToggle-dark.svg
View File

@ -1,5 +0,0 @@
<!-- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/. -->
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"
fill="rgba(255,255,255,1)"><path d="M3 1h10a3.008 3.008 0 0 1 3 3v8a3.009 3.009 0 0 1-3 3H3a3.005 3.005 0 0 1-3-3V4a3.013 3.013 0 0 1 3-3zm11 11V4a1 1 0 0 0-1-1H8v10h5a1 1 0 0 0 1-1zM2 12a1 1 0 0 0 1 1h4V3H3a1 1 0 0 0-1 1v8z"></path><path d="M3.5 5h2a.5.5 0 0 0 0-1h-2a.5.5 0 0 0 0 1zm0 2h2a.5.5 0 0 0 0-1h-2a.5.5 0 0 0 0 1zm1 2h1a.5.5 0 0 0 0-1h-1a.5.5 0 0 0 0 1z"></path></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 674 B

2
cps/static/css/libs/images/toolbarButton-viewAttachments-dark.svg
View File

@ -1,2 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16"
fill="rgba(255,255,255,1)"><path d="M6.2 2s.5-.5 1.06 0c.5.5 0 1 0 1l-4.6 4.61s-2.5 2.5 0 5 5 0 5 0L13.8 6.4s1.6-1.6 0-3.2-3.2 0-3.2 0L5.8 8s-.7.7 0 1.4 1.4 0 1.4 0l3.9-3.9s.6-.5 1 0c.5.5 0 1 0 1l-3.8 4s-1.8 1.8-3.5 0C3 8.7 4.8 7 4.8 7l4.7-4.9s2.7-2.6 5.3 0c2.6 2.6 0 5.3 0 5.3l-6.2 6.3s-3.5 3.5-7 0 0-7 0-7z"/></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 380 B

Some files were not shown because too many files have changed in this diff Show More