Change order of imports for goodreads to make import error message clear agan

Bugfix for too new lxml

.github/ISSUE_TEMPLATE/bug_report.md

@@ -6,12 +6,23 @@ labels: ''
 assignees: ''
 
 ---
-<!-- Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md) -->
 
-**Describe the bug/problem**
+## Short Notice from the maintainer 
+
+After 6 years of more or less intensive programming on Calibre-Web, I need a break. 
+The last few months, maintaining Calibre-Web has felt more like work than a hobby. I felt pressured and teased by people to solve "their" problems and merge PRs for "their" Calibre-Web. 
+I have turned off all notifications from Github/Discord and will now concentrate undisturbed on the development of “my” Calibre-Web over the next few weeks/months.  
+I will look into the issues and maybe also the PRs from time to time, but don't expect a quick response from me.
+
+
+Please also have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md)
+
+**Describe the bug/problem** 
+
 A clear and concise description of what the bug is. If you are asking for support, please check our [Wiki](https://github.com/janeczku/calibre-web/wiki) if your question is already answered there.
 
 **To Reproduce**
+
 Steps to reproduce the behavior:
 1. Go to '...'
 2. Click on '....'
@@ -19,15 +30,19 @@ Steps to reproduce the behavior:
 4. See error
 
 **Logfile**
+
 Add content of calibre-web.log file or the relevant error, try to reproduce your problem with "debug" log-level to get more output.
 
 **Expected behavior**
+
 A clear and concise description of what you expected to happen.
 
 **Screenshots**
+
 If applicable, add screenshots to help explain your problem.
 
 **Environment (please complete the following information):**
+
  - OS: [e.g. Windows 10/Raspberry Pi OS]
  - Python version: [e.g. python2.7]
  - Calibre-Web version: [e.g. 0.6.8 or 087c4c59 (git rev-parse --short HEAD)]:
@@ -37,3 +52,4 @@ If applicable, add screenshots to help explain your problem.
 
 **Additional context**
 Add any other context about the problem here. [e.g. access via reverse proxy, database background sync, special database location]
+

.github/ISSUE_TEMPLATE/feature_request.md

@@ -7,7 +7,14 @@ assignees: ''
 
 ---
 
-<!-- Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md) -->
+# Short Notice from the maintainer 
+
+After 6 years of more or less intensive programming on Calibre-Web, I need a break. 
+The last few months, maintaining Calibre-Web has felt more like work than a hobby. I felt pressured and teased by people to solve "their" problems and merge PRs for "their" Calibre-Web. 
+I have turned off all notifications from Github/Discord and will now concentrate undisturbed on the development of “my” Calibre-Web over the next few weeks/months.  
+I will look into the issues and maybe also the PRs from time to time, but don't expect a quick response from me.
+
+Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md)
 
 **Is your feature request related to a problem? Please describe.**
 A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

README.md

@@ -1,109 +1,125 @@
-# About
+# Short Notice from the maintainer 
 
-Calibre-Web is a web app providing a clean interface for browsing, reading and downloading eBooks using a valid [Calibre](https://calibre-ebook.com) database.
+After 6 years of more or less intensive programming on Calibre-Web, I need a break. 
+The last few months, maintaining Calibre-Web has felt more like work than a hobby. I felt pressured and teased by people to solve "their" problems and merge PRs for "their" Calibre-Web. 
+I have turned off all notifications from Github/Discord and will now concentrate undisturbed on the development of “my” Calibre-Web over the next few weeks/months.  
+I will look into the issues and maybe also the PRs from time to time, but don't expect a quick response from me.
 
-[![GitHub License](https://img.shields.io/github/license/janeczku/calibre-web?style=flat-square)](https://github.com/janeczku/calibre-web/blob/master/LICENSE)
-[![GitHub commit activity](https://img.shields.io/github/commit-activity/w/janeczku/calibre-web?logo=github&style=flat-square&label=commits)]()
-[![GitHub all releases](https://img.shields.io/github/downloads/janeczku/calibre-web/total?logo=github&style=flat-square)](https://github.com/janeczku/calibre-web/releases)
+# Calibre-Web
+
+Calibre-Web is a web app that offers a clean and intuitive interface for browsing, reading, and downloading eBooks using a valid [Calibre](https://calibre-ebook.com) database.
+
+[![License](https://img.shields.io/github/license/janeczku/calibre-web?style=flat-square)](https://github.com/janeczku/calibre-web/blob/master/LICENSE)
+![Commit Activity](https://img.shields.io/github/commit-activity/w/janeczku/calibre-web?logo=github&style=flat-square&label=commits)
+[![All Releases](https://img.shields.io/github/downloads/janeczku/calibre-web/total?logo=github&style=flat-square)](https://github.com/janeczku/calibre-web/releases)
 [![PyPI](https://img.shields.io/pypi/v/calibreweb?logo=pypi&logoColor=fff&style=flat-square)](https://pypi.org/project/calibreweb/)
 [![PyPI - Downloads](https://img.shields.io/pypi/dm/calibreweb?logo=pypi&logoColor=fff&style=flat-square)](https://pypi.org/project/calibreweb/)
 [![Discord](https://img.shields.io/discord/838810113564344381?label=Discord&logo=discord&style=flat-square)](https://discord.gg/h2VsJ2NEfB)
 
+<details>
+<summary><strong>Table of Contents</strong> (click to expand)</summary>
+
+1. [About](#calibre-web)
+2. [Features](#features)
+3. [Installation](#installation)
+   - [Installation via pip (recommended)](#installation-via-pip-recommended)
+   - [Quick start](#quick-start)
+   - [Requirements](#requirements)
+4. [Docker Images](#docker-images)
+5. [Contributor Recognition](#contributor-recognition)
+6. [Contact](#contact)
+7. [Contributing to Calibre-Web](#contributing-to-calibre-web)
+
+</details>
+
+
 *This software is a fork of [library](https://github.com/mutschler/calibreserver) and licensed under the GPL v3 License.*
 
 ![Main screen](https://github.com/janeczku/calibre-web/wiki/images/main_screen.png)
 
 ## Features
 
-- Bootstrap 3 HTML5 interface
-- full graphical setup
-- User management with fine-grained per-user permissions
+- Modern and responsive Bootstrap 3 HTML5 interface
+- Full graphical setup
+- Comprehensive user management with fine-grained per-user permissions
 - Admin interface
-- User Interface in brazilian, czech, dutch, english, finnish, french, galician, german, greek, hungarian, indonesian, italian, japanese, khmer, korean, norwegian, polish, russian, simplified and traditional chinese, spanish, swedish, turkish, ukrainian, vietnamese
-- OPDS feed for eBook reader apps 
-- Filter and search by titles, authors, tags, series, book format and language
-- Create a custom book collection (shelves)
-- Support for editing eBook metadata and deleting eBooks from Calibre library
-- Support for downloading eBook metadata from various sources, sources can be extended via external plugins
-- Support for converting eBooks through Calibre binaries
-- Restrict eBook download to logged-in users
-- Support for public user registration
-- Send eBooks to E-Readers with the click of a button
-- Sync your Kobo devices through Calibre-Web with your Calibre library
-- Support for reading eBooks directly in the browser (.txt, .epub, .pdf, .cbr, .cbt, .cbz, .djvu)
-- Upload new books in many formats, including audio formats (.mp3, .m4a, .m4b)
-- Support for Calibre Custom Columns
-- Ability to hide content based on categories and Custom Column content per user
+- Multilingual user interface supporting 20+ languages ([supported languages](https://github.com/janeczku/calibre-web/wiki/Translation-Status))
+- OPDS feed for eBook reader apps
+- Advanced search and filtering options
+- Custom book collection (shelves) creation
+- eBook metadata editing and deletion support
+- Metadata download from various sources (extensible via plugins)
+- eBook conversion through Calibre binaries
+- eBook download restriction to logged-in users
+- Public user registration support
+- Send eBooks to E-Readers with a single click
+- Sync Kobo devices with your Calibre library
+- In-browser eBook reading support for multiple formats
+- Upload new books in various formats, including audio formats
+- Calibre Custom Columns support
+- Content hiding based on categories and Custom Column content per user
 - Self-update capability
-- "Magic Link" login to make it easy to log on eReaders
-- Login via LDAP, google/github oauth and via proxy authentication
+- "Magic Link" login for easy access on eReaders
+- LDAP, Google/GitHub OAuth, and proxy authentication support
 
 ## Installation
 
 #### Installation via pip (recommended)
-1. To avoid problems with already installed python dependencies, it's recommended to create a virtual environment for Calibre-Web
-2. Install Calibre-Web via pip with the command `pip install calibreweb` (Depending on your OS and or distro the command could also be `pip3`). 
-3. Optional features can also be installed via pip, please refer to [this page](https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-and-Windows) for details 
-4. Calibre-Web can be started afterwards by typing `cps` 
+1. Create a virtual environment for Calibre-Web to avoid conflicts with existing Python dependencies
+2. Install Calibre-Web via pip: `pip install calibreweb` (or `pip3` depending on your OS/distro)
+3. Install optional features via pip as needed, see [this page](https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-and-Windows) for details
+4. Start Calibre-Web by typing `cps`
 
-Issues with Raspberry Pi - Raspberry Pi OS:
-Depending on your version of pip it's possible that the installation fails with `Failed to build cryptography
-ERROR: Could not build wheels for cryptography, which is required to install pyproject.toml-based projects`. 
-In this case please try to update pip with `./venv/bin/python3 -m pip install --upgrade pip` first, and then try installing Calibre-Web again. 
-If this isn't working please also install cargo via `sudo apt install cargo`, and try installing Calibre-Web again.
+*Note: Raspberry Pi OS users may encounter issues during installation. If so, please update pip (`./venv/bin/python3 -m pip install --upgrade pip`) and/or install cargo (`sudo apt install cargo`) before retrying the installation.*
 
-In the Wiki there are also examples for: a [manual installation](https://github.com/janeczku/calibre-web/wiki/Manual-installation), [installation on Linux Mint](https://github.com/janeczku/calibre-web/wiki/How-To:Install-Calibre-Web-in-Linux-Mint-19-or-20), [installation on a Cloud Provider](https://github.com/janeczku/calibre-web/wiki/How-To:-Install-Calibre-Web-on-a-Cloud-Provider).
+Refer to the Wiki for additional installation examples: [manual installation](https://github.com/janeczku/calibre-web/wiki/Manual-installation), [Linux Mint](https://github.com/janeczku/calibre-web/wiki/How-To:-Install-Calibre-Web-in-Linux-Mint-19-or-20), [Cloud Provider](https://github.com/janeczku/calibre-web/wiki/How-To:-Install-Calibre-Web-on-a-Cloud-Provider).
 
-## Quick start
+## Quick Start
 
-Point your browser to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog \
-Login with default admin login \
-If you don't have a Calibre database already, this [database](https://github.com/janeczku/calibre-web/blob/master/library/metadata.db) can be used. **IMPORTATNT** Please move the database out of the calibre-web folder structure, as it will be overwritten during update. \
-Set `Location of Calibre database` to the path of the folder where your Calibre library (metadata.db) lives, push "submit" button. \
-Optionally a Google Drive can be used to host the calibre library [-> Using Google Drive integration](https://github.com/janeczku/calibre-web/wiki/G-Drive-Setup#using-google-drive-integration) \
-Afterwards you can configure your Calibre-Web instance ([Basic Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#basic-configuration) and [UI Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#ui-configuration) on admin page)  
-
-#### Default admin login:
-*Username:* admin\
-*Password:* admin123
+1. Open your browser and navigate to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog
+2. Log in with the default admin credentials
+3. If you don't have a Calibre database, you can use [this database](https://github.com/janeczku/calibre-web/raw/master/library/metadata.db) (move it out of the Calibre-Web folder to prevent overwriting during updates)
+4. Set `Location of Calibre database` to the path of the folder containing your Calibre library (metadata.db) and click "Save"
+5. Optionally, use Google Drive to host your Calibre library by following the [Google Drive integration guide](https://github.com/janeczku/calibre-web/wiki/G-Drive-Setup#using-google-drive-integration)
+6. Configure your Calibre-Web instance via the admin page, referring to the [Basic Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#basic-configuration) and [UI Configuration](https://github.com/janeczku/calibre-web/wiki/Configuration#ui-configuration) guides
 
+#### Default Admin Login:
+- **Username:** admin
+- **Password:** admin123
 
 ## Requirements
 
-python 3.5+
-
-[Download](https://imagemagick.org/script/download.php) Imagemagick to extract covers from epubs. On Windows the additional installation of [ghostscript](https://ghostscript.com/releases/gsdnld.html) might be necessary to extract covers from pdf files. On Linux Imagemagick and Ghostscript can often be installed using the system package manager.
-
-Optionally, to enable on-the-fly conversion from one ebook format to another when using the send-to-ereader feature, or during editing of ebooks metadata:
-
-[Download and install](https://calibre-ebook.com/download) the Calibre desktop program for your platform and enter the folder including program name (normally /opt/calibre/ebook-convert, or C:\Program Files\calibre\ebook-convert.exe) in the field "calibre's converter tool" on the setup page.
-
-[Download](https://github.com/pgaskin/kepubify/releases/latest) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `/opt/kepubify` Windows: `C:\Program Files\kepubify`.
-
+- Python 3.5+
+- [Imagemagick](https://imagemagick.org/script/download.php) for cover extraction from EPUBs (Windows users may need to install [Ghostscript](https://ghostscript.com/releases/gsdnld.html) for PDF cover extraction)
+- Optional: [Calibre desktop program](https://calibre-ebook.com/download) for on-the-fly conversion and metadata editing (set "calibre's converter tool" path on the setup page)
+- Optional: [Kepubify tool](https://github.com/pgaskin/kepubify/releases/latest) for Kobo device support (place the binary in `/opt/kepubify` on Linux or `C:\Program Files\kepubify` on Windows)
 
 ## Docker Images
 
-A pre-built Docker image is available in these Docker Hub repository (maintained by the LinuxServer team):
+Pre-built Docker images are available in the following Docker Hub repositories (maintained by the LinuxServer team):
 
-#### **LinuxServer - x64, armhf, aarch64**
-+ Docker Hub - [https://hub.docker.com/r/linuxserver/calibre-web](https://hub.docker.com/r/linuxserver/calibre-web)
-+ Github - [https://github.com/linuxserver/docker-calibre-web](https://github.com/linuxserver/docker-calibre-web)
-+ Github - (Optional Calibre layer) - [https://github.com/linuxserver/docker-calibre-web/tree/calibre](https://github.com/linuxserver/docker-calibre-web/tree/calibre) 
+#### **LinuxServer - x64, aarch64**
+- [Docker Hub](https://hub.docker.com/r/linuxserver/calibre-web)
+- [GitHub](https://github.com/linuxserver/docker-calibre-web)
+- [GitHub - Optional Calibre layer](https://github.com/linuxserver/docker-mods/tree/universal-calibre)
 
-   This image has the option to pull in an extra docker manifest layer to include the Calibre `ebook-convert` binary.  Just include the environmental variable `DOCKER_MODS=linuxserver/calibre-web:calibre` in your docker run/docker compose file. **(x64 only)**
-  
-   If you do not need this functionality then this can be omitted, keeping the image as lightweight as possible.
-    
-   Both the Calibre-Web and Calibre-Mod images are rebuilt automatically on new releases of Calibre-Web and Calibre respectively, and on updates to any included base image packages on a weekly basis if required.
-   + The "path to convertertool" should be set to `/usr/bin/ebook-convert`
-   + The "path to unrar" should be set to `/usr/bin/unrar`
+  Include the environment variable `DOCKER_MODS=linuxserver/mods:universal-calibre` in your Docker run/compose file to add the Calibre `ebook-convert` binary (x64 only). Omit this variable for a lightweight image.
 
-# Contact
+  Both the Calibre-Web and Calibre-Mod images are automatically rebuilt on new releases and updates.
 
-Just reach us out on [Discord](https://discord.gg/h2VsJ2NEfB)
+  - Set "path to convertertool" to `/usr/bin/ebook-convert`
+  - Set "path to unrar" to `/usr/bin/unrar`
 
-For further information, How To's and FAQ please check the [Wiki](https://github.com/janeczku/calibre-web/wiki)
+## Contributor Recognition
 
-# Contributing to Calibre-Web
+We would like to thank all the [contributors](https://github.com/janeczku/calibre-web/graphs/contributors) and maintainers of Calibre-Web for their valuable input and dedication to the project. Your contributions are greatly appreciated.
 
-Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md) 
+## Contact
+
+Join us on [Discord](https://discord.gg/h2VsJ2NEfB)
+
+For more information, How To's, and FAQs, please visit the [Wiki](https://github.com/janeczku/calibre-web/wiki)
+
+## Contributing to Calibre-Web
+
+Check out our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md)

SECURITY.md

@@ -38,6 +38,13 @@ To receive fixes for security vulnerabilities it is required to always upgrade t
 | V 0.6.18      | Possible SQL Injection is prevented in user table  Thanks to Iman Sharafaldin (Forward Security)                   |CVE-2022-30765|
 | V 0.6.18      | The SSRF protection no longer can be bypassed by IPV6/IPV4 embedding. Thanks to  @416e6e61                         |CVE-2022-0939|
 | V 0.6.18      | The SSRF protection no longer can be bypassed to connect to other servers in the local network. Thanks to @michaellrowley |CVE-2022-0990|
+| V 0.6.20      | Credentials for emails are now stored encrypted ||
+| V 0.6.20      |  Login is rate limited ||
+| V 0.6.20      |  Passwordstrength can be forced ||
+| V 0.6.21      |  SMTP server credentials are no longer returned to client ||
+| V 0.6.21      |  Cross-site scripting (XSS) stored in href bypasses filter using data wrapper no longer possible ||
+| V 0.6.21      |  Cross-site scripting (XSS) is no longer possible via pathchooser ||
+| V 0.6.21      |  Error Handling at non existent rating, language, and user downloaded books was fixed ||
 
 
 ## Statement regarding Log4j (CVE-2021-44228 and related)

cps/MyLoginManager.py

@@ -28,10 +28,10 @@ from flask_login.signals import user_loaded_from_cookie
 
 class MyLoginManager(LoginManager):
     def _session_protection_failed(self):
-        _session = session._get_current_object()
+        sess = session._get_current_object()
         ident = self._session_identifier_generator()
-        if(_session and not (len(_session) == 1
-                             and _session.get('csrf_token', None))) and ident != _session.get('_id', None):
+        if(sess and not (len(sess) == 1
+                             and sess.get('csrf_token', None))) and ident != sess.get('_id', None):
             return super(). _session_protection_failed()
         return False
 

cps/__init__.py

@@ -64,7 +64,8 @@ mimetypes.add_type('application/x-mobi8-ebook', '.azw3')
 mimetypes.add_type('application/x-cbr', '.cbr')
 mimetypes.add_type('application/x-cbz', '.cbz')
 mimetypes.add_type('application/x-cbt', '.cbt')
-mimetypes.add_type('image/vnd.djvu', '.djvu')
+mimetypes.add_type('application/x-cb7', '.cb7')
+mimetypes.add_type('image/vnd.djv', '.djv')
 mimetypes.add_type('application/mpeg', '.mpeg')
 mimetypes.add_type('application/mpeg', '.mp3')
 mimetypes.add_type('application/mp4', '.m4a')
@@ -86,10 +87,10 @@ app.config.update(
 
 lm = MyLoginManager()
 
-config = config_sql.ConfigSQL()
-
 cli_param = CliParameter()
 
+config = config_sql.ConfigSQL()
+
 if wtf_present:
     csrf = CSRFProtect()
 else:
@@ -102,7 +103,7 @@ web_server = WebServer()
 updater_thread = Updater()
 
 if limiter_present:
-    limiter = Limiter(key_func=True, headers_enabled=True, auto_check=False, swallow_errors=True)
+    limiter = Limiter(key_func=True, headers_enabled=True, auto_check=False, swallow_errors=False)
 else:
     limiter = None
 
@@ -124,13 +125,6 @@ def create_app():
 
     ub.password_change(cli_param.user_credentials)
 
-    if not limiter:
-        log.info('*** "flask-limiter" is needed for calibre-web to run. '
-                 'Please install it using pip: "pip install flask-limiter" ***')
-        print('*** "flask-limiter" is needed for calibre-web to run. '
-              'Please install it using pip: "pip install flask-limiter" ***')
-        web_server.stop(True)
-        sys.exit(8)
     if sys.version_info < (3, 0):
         log.info(
             '*** Python2 is EOL since end of 2019, this version of Calibre-Web is no longer supporting Python2, '
@@ -140,13 +134,6 @@ def create_app():
             'please update your installation to Python3 ***')
         web_server.stop(True)
         sys.exit(5)
-    if not wtf_present:
-        log.info('*** "flask-WTF" is needed for calibre-web to run. '
-                 'Please install it using pip: "pip install flask-WTF" ***')
-        print('*** "flask-WTF" is needed for calibre-web to run. '
-              'Please install it using pip: "pip install flask-WTF" ***')
-        web_server.stop(True)
-        sys.exit(7)
 
     lm.login_view = 'web.login'
     lm.anonymous_user = ub.Anonymous
@@ -157,13 +144,21 @@ def create_app():
     calibre_db.init_db()
 
     updater_thread.init_updater(config, web_server)
-    # Perform dry run of updater and exit afterwards
+    # Perform dry run of updater and exit afterward
     if cli_param.dry_run:
         updater_thread.dry_run()
         sys.exit(0)
     updater_thread.start()
-
-    for res in dependency_check() + dependency_check(True):
+    requirements = dependency_check()
+    for res in requirements:
+        if res['found'] == "not installed":
+            message = ('Cannot import {name} module, it is needed to run calibre-web, '
+                       'please install it using "pip install {name}"').format(name=res["name"])
+            log.info(message)
+            print("*** " + message + " ***")
+            web_server.stop(True)
+            sys.exit(8)
+    for res in requirements + dependency_check(True):
         log.info('*** "{}" version does not meet the requirements. '
                  'Should: {}, Found: {}, please consider installing required version ***'
                  .format(res['name'],
@@ -191,12 +186,21 @@ def create_app():
         services.ldap.init_app(app, config)
     if services.goodreads_support:
         services.goodreads_support.connect(config.config_goodreads_api_key,
-                                           config.config_goodreads_api_secret_e,
                                            config.config_use_goodreads)
     config.store_calibre_uuid(calibre_db, db.Library_Id)
     # Configure rate limiter
+    # https://limits.readthedocs.io/en/stable/storage.html
     app.config.update(RATELIMIT_ENABLED=config.config_ratelimiter)
-    limiter.init_app(app)
+    if config.config_limiter_uri != "" and not cli_param.memory_backend:
+        app.config.update(RATELIMIT_STORAGE_URI=config.config_limiter_uri)
+        if config.config_limiter_options != "":
+            app.config.update(RATELIMIT_STORAGE_OPTIONS=config.config_limiter_options)
+    try:
+        limiter.init_app(app)
+    except Exception as e:
+        log.error('Wrong Flask Limiter configuration, falling back to default: {}'.format(e))
+        app.config.update(RATELIMIT_STORAGE_URI=None)
+        limiter.init_app(app)
 
     # Register scheduled tasks
     from .schedule import register_scheduled_tasks, register_startup_tasks

cps/about.py

@@ -49,9 +49,9 @@ sorted_modules = OrderedDict((sorted(modules.items(), key=lambda x: x[0].casefol
 
 def collect_stats():
     if constants.NIGHTLY_VERSION[0] == "$Format:%H$":
-        calibre_web_version = constants.STABLE_VERSION['version']
+        calibre_web_version = constants.STABLE_VERSION['version'].replace("b", " Beta")
     else:
-        calibre_web_version = (constants.STABLE_VERSION['version'] + ' - '
+        calibre_web_version = (constants.STABLE_VERSION['version'].replace("b", " Beta") + ' - '
                                + constants.NIGHTLY_VERSION[0].replace('%', '%%') + ' - '
                                + constants.NIGHTLY_VERSION[1].replace('%', '%%'))
 

cps/admin.py

@@ -33,6 +33,7 @@ from functools import wraps
 from urllib.parse import urlparse
 
 from flask import Blueprint, flash, redirect, url_for, abort, request, make_response, send_from_directory, g, Response
+from markupsafe import Markup
 from flask_login import login_required, current_user, logout_user
 from flask_babel import gettext as _
 from flask_babel import get_locale, format_time, format_datetime, format_timedelta
@@ -47,6 +48,7 @@ from . import db, calibre_db, ub, web_server, config, updater_thread, gdriveutil
     kobo_sync_status, schedule
 from .helper import check_valid_domain, send_test_mail, reset_password, generate_password_hash, check_email, \
     valid_email, check_username
+from .embed_helper import get_calibre_binarypath
 from .gdriveutils import is_gdrive_ready, gdrive_support
 from .render_template import render_title_template, get_sidebar_config
 from .services.worker import WorkerThread
@@ -101,10 +103,15 @@ def admin_required(f):
 
 @admi.before_app_request
 def before_request():
-    if not ub.check_user_session(current_user.id, flask_session.get('_id')) and 'opds' not in request.path:
-        logout_user()
+    try:
+        if not ub.check_user_session(current_user.id,
+                                     flask_session.get('_id')) and 'opds' not in request.path \
+          and config.config_session == 1:
+            logout_user()
+    except AttributeError:
+        pass    # ? fails on requesting /ajax/emailstat during restart ?
     g.constants = constants
-    g.google_site_verification = os.getenv('GOOGLE_SITE_VERIFICATION','')
+    g.google_site_verification = os.getenv('GOOGLE_SITE_VERIFICATION', '')
     g.allow_registration = config.config_public_reg
     g.allow_anonymous = config.config_anonbrowse
     g.allow_upload = config.config_uploading
@@ -211,7 +218,7 @@ def admin():
                     form_date += timedelta(hours=int(commit[20:22]), minutes=int(commit[23:]))
             commit = format_datetime(form_date - tz, format='short')
         else:
-            commit = version['version']
+            commit = version['version'].replace("b", " Beta")
 
     all_user = ub.session.query(ub.User).all()
     # email_settings = mail_config.get_mail_settings()
@@ -910,11 +917,15 @@ def list_restriction(res_type, user_id):
 
 @admi.route("/ajax/fullsync", methods=["POST"])
 @login_required
-def ajax_fullsync():
-    count = ub.session.query(ub.KoboSyncedBooks).filter(current_user.id == ub.KoboSyncedBooks.user_id).delete()
-    message = _("{} sync entries deleted").format(count)
-    ub.session_commit(message)
-    return Response(json.dumps([{"type": "success", "message": message}]), mimetype='application/json')
+def ajax_self_fullsync():
+    return do_full_kobo_sync(current_user.id)
+
+
+@admi.route("/ajax/fullsync/<int:userid>", methods=["POST"])
+@login_required
+@admin_required
+def ajax_fullsync(userid):
+    return do_full_kobo_sync(userid)
 
 
 @admi.route("/ajax/pathchooser/")
@@ -924,6 +935,13 @@ def ajax_pathchooser():
     return pathchooser()
 
 
+def do_full_kobo_sync(userid):
+    count = ub.session.query(ub.KoboSyncedBooks).filter(userid == ub.KoboSyncedBooks.user_id).delete()
+    message = _("{} sync entries deleted").format(count)
+    ub.session_commit(message)
+    return Response(json.dumps([{"type": "success", "message": message}]), mimetype='application/json')
+
+
 def check_valid_read_column(column):
     if column != "0":
         if not calibre_db.session.query(db.CustomColumns).filter(db.CustomColumns.id == column) \
@@ -1033,7 +1051,8 @@ def pathchooser():
 
     for f in folders:
         try:
-            data = {"name": f, "fullpath": os.path.join(cwd, f)}
+            sanitized_f = str(Markup.escape(f))
+            data = {"name": sanitized_f, "fullpath": os.path.join(cwd, sanitized_f)}
             data["sort"] = data["fullpath"].lower()
         except Exception:
             continue
@@ -1292,7 +1311,8 @@ def update_mailsettings():
     else:
         _config_int(to_save, "mail_port")
         _config_int(to_save, "mail_use_ssl")
-        _config_string(to_save, "mail_password_e")
+        if to_save.get("mail_password_e", ""):
+            _config_string(to_save, "mail_password_e")
         _config_int(to_save, "mail_size", lambda y: int(y) * 1024 * 1024)
         config.mail_server = to_save.get('mail_server', "").strip()
         config.mail_from = to_save.get('mail_from', "").strip()
@@ -1611,7 +1631,10 @@ def import_ldap_users():
 
     imported = 0
     for username in new_users:
-        user = username.decode('utf-8')
+        if isinstance(username, bytes):
+            user = username.decode('utf-8')
+        else:
+            user = username
         if '=' in user:
             # if member object field is empty take user object as filter
             if config.config_ldap_member_user_object:
@@ -1720,6 +1743,9 @@ def _db_configuration_update_helper():
         calibre_db.update_config(config)
         if not os.access(os.path.join(config.config_calibre_dir, "metadata.db"), os.W_OK):
             flash(_("DB is not Writeable"), category="warning")
+    _config_string(to_save, "config_calibre_split_dir")
+    config.config_calibre_split = to_save.get('config_calibre_split', 0) == "on"
+    calibre_db.update_config(config)
     config.save()
     return _db_configuration_result(None, gdrive_error)
 
@@ -1740,6 +1766,7 @@ def _configuration_update_helper():
 
         _config_checkbox_int(to_save, "config_uploading")
         _config_checkbox_int(to_save, "config_unicode_filename")
+        _config_checkbox_int(to_save, "config_embed_metadata")
         # Reboot on config_anonbrowse with enabled ldap, as decoraters are changed in this case
         reboot_required |= (_config_checkbox_int(to_save, "config_anonbrowse")
                             and config.config_login_type == constants.LOGIN_LDAP)
@@ -1756,8 +1783,14 @@ def _configuration_update_helper():
             constants.EXTENSIONS_UPLOAD = config.config_upload_formats.split(',')
 
         _config_string(to_save, "config_calibre")
-        _config_string(to_save, "config_converterpath")
+        _config_string(to_save, "config_binariesdir")
         _config_string(to_save, "config_kepubifypath")
+        if "config_binariesdir" in to_save:
+            calibre_status = helper.check_calibre(config.config_binariesdir)
+            if calibre_status:
+                return _configuration_result(calibre_status)
+            to_save["config_converterpath"] = get_calibre_binarypath("ebook-convert")
+            _config_string(to_save, "config_converterpath")
 
         reboot_required |= _config_int(to_save, "config_login_type")
 
@@ -1776,10 +1809,8 @@ def _configuration_update_helper():
         # Goodreads configuration
         _config_checkbox(to_save, "config_use_goodreads")
         _config_string(to_save, "config_goodreads_api_key")
-        _config_string(to_save, "config_goodreads_api_secret_e")
         if services.goodreads_support:
             services.goodreads_support.connect(config.config_goodreads_api_key,
-                                               config.config_goodreads_api_secret_e,
                                                config.config_use_goodreads)
 
         _config_int(to_save, "config_updatechannel")
@@ -1803,6 +1834,7 @@ def _configuration_update_helper():
         _config_checkbox(to_save, "config_password_number")
         _config_checkbox(to_save, "config_password_lower")
         _config_checkbox(to_save, "config_password_upper")
+        _config_checkbox(to_save, "config_password_character")
         _config_checkbox(to_save, "config_password_special")
         if 0 < int(to_save.get("config_password_min_length", "0")) < 41:
             _config_int(to_save, "config_password_min_length")
@@ -1810,6 +1842,8 @@ def _configuration_update_helper():
             return _configuration_result(_('Password length has to be between 1 and 40'))
         reboot_required |= _config_int(to_save, "config_session")
         reboot_required |= _config_checkbox(to_save, "config_ratelimiter")
+        reboot_required |= _config_string(to_save, "config_limiter_uri")
+        reboot_required |= _config_string(to_save, "config_limiter_options")
 
         # Rarfile Content configuration
         _config_string(to_save, "config_rarfile_location")

cps/clean_html.py

@@ -0,0 +1,53 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2018-2019 OzzieIsaacs
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from . import logger
+from lxml.etree import ParserError
+
+try:
+    # at least bleach 6.0 is needed -> incomplatible change from list arguments to set arguments
+    from bleach import clean_text as clean_html
+    BLEACH = True
+except ImportError:
+    try:
+        BLEACH = False
+        from nh3 import clean as clean_html
+    except ImportError:
+        try:
+            BLEACH = False
+            from lxml.html.clean import clean_html
+        except ImportError:
+            clean_html = None
+
+
+log = logger.create()
+
+
+def clean_string(unsafe_text, book_id=0):
+    try:
+        if BLEACH:
+            safe_text = clean_html(unsafe_text, tags=set(), attributes=set())
+        else:
+            safe_text = clean_html(unsafe_text)
+    except ParserError as e:
+        log.error("Comments of book {} are corrupted: {}".format(book_id, e))
+        safe_text = ""
+    except TypeError as e:
+        log.error("Comments can't be parsed, maybe 'lxml' is too new, try installing 'bleach': {}".format(e))
+        safe_text = ""
+    return safe_text

cps/cli.py

@@ -29,8 +29,8 @@ from .constants import DEFAULT_SETTINGS_FILE, DEFAULT_GDRIVE_FILE
 
 def version_info():
     if _NIGHTLY_VERSION[1].startswith('$Format'):
-        return "Calibre-Web version: %s - unknown git-clone" % _STABLE_VERSION['version']
-    return "Calibre-Web version: %s -%s" % (_STABLE_VERSION['version'], _NIGHTLY_VERSION[1])
+        return "Calibre-Web version: %s - unknown git-clone" % _STABLE_VERSION['version'].replace("b", " Beta")
+    return "Calibre-Web version: %s -%s" % (_STABLE_VERSION['version'].replace("b", " Beta"), _NIGHTLY_VERSION[1])
 
 
 class CliParameter(object):
@@ -48,9 +48,11 @@ class CliParameter(object):
                                                        'works only in combination with keyfile')
         parser.add_argument('-k', metavar='path', help='path and name to SSL keyfile, e.g. /opt/test.key, '
                                                        'works only in combination with certfile')
+        parser.add_argument('-o', metavar='path', help='path and name Calibre-Web logfile')
         parser.add_argument('-v', '--version', action='version', help='Shows version number and exits Calibre-Web',
                             version=version_info())
         parser.add_argument('-i', metavar='ip-address', help='Server IP-Address to listen')
+        parser.add_argument('-m', action='store_true', help='Use Memory-backend as limiter backend, use this parameter in case of miss configured backend')
         parser.add_argument('-s', metavar='user:pass',
                             help='Sets specific username to new password and exits Calibre-Web')
         parser.add_argument('-f', action='store_true', help='Flag is depreciated and will be removed in next version')
@@ -60,6 +62,7 @@ class CliParameter(object):
         parser.add_argument('-r', action='store_true', help='Enable public database reconnect route under /reconnect')
         args = parser.parse_args()
 
+        self.logpath = args.o or ""
         self.settings_path = args.p or os.path.join(_CONFIG_DIR, DEFAULT_SETTINGS_FILE)
         self.gd_path = args.g or os.path.join(_CONFIG_DIR, DEFAULT_GDRIVE_FILE)
 
@@ -96,6 +99,8 @@ class CliParameter(object):
         if args.k == "":
             self.keyfilepath = ""
 
+        # overwrite limiter backend
+        self.memory_backend = args.m or None
         # dry run updater
         self.dry_run = args.d or None
         # enable reconnect endpoint for docker database reconnect

cps/comic.py

@@ -36,6 +36,12 @@ try:
         from comicapi import __version__ as comic_version
     except ImportError:
         comic_version = ''
+    try:
+        from comicapi.comicarchive import load_archive_plugins
+        import comicapi.utils
+        comicapi.utils.add_rar_paths()
+    except ImportError:
+        load_archive_plugins = None
 except (ImportError, LookupError) as e:
     log.debug('Cannot import comicapi, extracting comic metadata will not work: %s', e)
     import zipfile
@@ -46,6 +52,12 @@ except (ImportError, LookupError) as e:
     except (ImportError, SyntaxError) as e:
         log.debug('Cannot import rarfile, extracting cover files from rar files will not work: %s', e)
         use_rarfile = False
+    try:
+        import py7zr
+        use_7zip = True
+    except (ImportError, SyntaxError) as e:
+        log.debug('Cannot import py7zr, extracting cover files from CB7 files will not work: %s', e)
+        use_7zip = False
     use_comic_meta = False
 
 
@@ -78,23 +90,40 @@ def _extract_cover_from_archive(original_file_extension, tmp_file_name, rar_exec
                 if len(ext) > 1:
                     extension = ext[1].lower()
                     if extension in cover.COVER_EXTENSIONS:
-                        cover_data = cf.read(name)
+                        cover_data = cf.read([name])
                         break
         except Exception as ex:
-            log.debug('Rarfile failed with error: {}'.format(ex))
+            log.error('Rarfile failed with error: {}'.format(ex))
+    elif original_file_extension.upper() == '.CB7' and use_7zip:
+        cf = py7zr.SevenZipFile(tmp_file_name)
+        for name in cf.getnames():
+            ext = os.path.splitext(name)
+            if len(ext) > 1:
+                extension = ext[1].lower()
+                if extension in cover.COVER_EXTENSIONS:
+                    try:
+                        cover_data = cf.read([name])[name].read()
+                    except (py7zr.Bad7zFile, OSError) as ex:
+                        log.error('7Zip file failed with error: {}'.format(ex))
+                    break
     return cover_data, extension
 
 
 def _extract_cover(tmp_file_name, original_file_extension, rar_executable):
     cover_data = extension = None
     if use_comic_meta:
-        archive = ComicArchive(tmp_file_name, rar_exe_path=rar_executable)
-        for index, name in enumerate(archive.getPageNameList()):
+        try:
+            archive = ComicArchive(tmp_file_name, rar_exe_path=rar_executable)
+        except TypeError:
+            archive = ComicArchive(tmp_file_name)
+        name_list = archive.getPageNameList if hasattr(archive, "getPageNameList") else archive.get_page_name_list
+        for index, name in enumerate(name_list()):
             ext = os.path.splitext(name)
             if len(ext) > 1:
                 extension = ext[1].lower()
                 if extension in cover.COVER_EXTENSIONS:
-                    cover_data = archive.getPage(index)
+                    get_page = archive.getPage if hasattr(archive, "getPageNameList") else archive.get_page
+                    cover_data = get_page(index)
                     break
     else:
         cover_data, extension = _extract_cover_from_archive(original_file_extension, tmp_file_name, rar_executable)
@@ -103,17 +132,26 @@ def _extract_cover(tmp_file_name, original_file_extension, rar_executable):
 
 def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rar_executable):
     if use_comic_meta:
-        archive = ComicArchive(tmp_file_path, rar_exe_path=rar_executable)
-        if archive.seemsToBeAComicArchive():
-            if archive.hasMetadata(MetaDataStyle.CIX):
+        try:
+            archive = ComicArchive(tmp_file_path, rar_exe_path=rar_executable)
+        except TypeError:
+            load_archive_plugins(force=True, rar=rar_executable)
+            archive = ComicArchive(tmp_file_path)
+        if hasattr(archive, "seemsToBeAComicArchive"):
+            seems_archive = archive.seemsToBeAComicArchive
+        else:
+            seems_archive = archive.seems_to_be_a_comic_archive
+        if seems_archive():
+            has_metadata = archive.hasMetadata if hasattr(archive, "hasMetadata") else archive.has_metadata
+            if has_metadata(MetaDataStyle.CIX):
                 style = MetaDataStyle.CIX
-            elif archive.hasMetadata(MetaDataStyle.CBI):
+            elif has_metadata(MetaDataStyle.CBI):
                 style = MetaDataStyle.CBI
             else:
                 style = None
 
-            # if style is not None:
-            loaded_metadata = archive.readMetadata(style)
+            read_metadata = archive.readMetadata if hasattr(archive, "readMetadata") else archive.read_metadata
+            loaded_metadata = read_metadata(style)
 
             lang = loaded_metadata.language or ""
             loaded_metadata.language = isoLanguages.get_lang3(lang)

cps/config_sql.py

@@ -34,6 +34,7 @@ except ImportError:
     from sqlalchemy.ext.declarative import declarative_base
 
 from . import constants, logger
+from .subproc_wrapper import process_wait
 
 
 log = logger.create()
@@ -69,6 +70,8 @@ class _Settings(_Base):
 
     config_calibre_dir = Column(String)
     config_calibre_uuid = Column(String)
+    config_calibre_split = Column(Boolean, default=False)
+    config_calibre_split_dir = Column(String)
     config_port = Column(Integer, default=constants.DEFAULT_PORT)
     config_external_port = Column(Integer, default=constants.DEFAULT_PORT)
     config_certfile = Column(String)
@@ -83,9 +86,9 @@ class _Settings(_Base):
     config_theme = Column(Integer, default=0)
 
     config_log_level = Column(SmallInteger, default=logger.DEFAULT_LOG_LEVEL)
-    config_logfile = Column(String)
+    config_logfile = Column(String, default=logger.DEFAULT_LOG_FILE)
     config_access_log = Column(SmallInteger, default=0)
-    config_access_logfile = Column(String)
+    config_access_logfile = Column(String, default=logger.DEFAULT_ACCESS_LOG)
 
     config_uploading = Column(SmallInteger, default=0)
     config_anonbrowse = Column(SmallInteger, default=0)
@@ -111,8 +114,6 @@ class _Settings(_Base):
 
     config_use_goodreads = Column(Boolean, default=False)
     config_goodreads_api_key = Column(String)
-    config_goodreads_api_secret_e = Column(String)
-    config_goodreads_api_secret = Column(String)
     config_register_email = Column(Boolean, default=False)
     config_login_type = Column(Integer, default=0)
 
@@ -138,10 +139,12 @@ class _Settings(_Base):
 
     config_kepubifypath = Column(String, default=None)
     config_converterpath = Column(String, default=None)
+    config_binariesdir = Column(String, default=None)
     config_calibre = Column(String)
     config_rarfile_location = Column(String, default=None)
     config_upload_formats = Column(String, default=','.join(constants.EXTENSIONS_UPLOAD))
     config_unicode_filename = Column(Boolean, default=False)
+    config_embed_metadata = Column(Boolean, default=True)
 
     config_updatechannel = Column(Integer, default=constants.UPDATE_STABLE)
 
@@ -160,9 +163,12 @@ class _Settings(_Base):
     config_password_number = Column(Boolean, default=True)
     config_password_lower = Column(Boolean, default=True)
     config_password_upper = Column(Boolean, default=True)
+    config_password_character = Column(Boolean, default=True)
     config_password_special = Column(Boolean, default=True)
     config_session = Column(Integer, default=1)
     config_ratelimiter = Column(Boolean, default=True)
+    config_limiter_uri = Column(String, default="")
+    config_limiter_options = Column(String, default="")
 
     def __repr__(self):
         return self.__class__.__name__
@@ -184,9 +190,11 @@ class ConfigSQL(object):
         self.load()
 
         change = False
-        if self.config_converterpath == None:  # pylint: disable=access-member-before-definition
+
+        if self.config_binariesdir == None: # pylint: disable=access-member-before-definition
             change = True
-            self.config_converterpath = autodetect_calibre_binary()
+            self.config_binariesdir = autodetect_calibre_binaries()
+            self.config_converterpath = autodetect_converter_binary(self.config_binariesdir)
 
         if self.config_kepubifypath == None:  # pylint: disable=access-member-before-definition
             change = True
@@ -341,14 +349,17 @@ class ConfigSQL(object):
             have_metadata_db = os.path.isfile(db_file)
         self.db_configured = have_metadata_db
         constants.EXTENSIONS_UPLOAD = [x.lstrip().rstrip().lower() for x in self.config_upload_formats.split(',')]
+        from . import cli_param
         if os.environ.get('FLASK_DEBUG'):
             logfile = logger.setup(logger.LOG_TO_STDOUT, logger.logging.DEBUG)
         else:
             # pylint: disable=access-member-before-definition
-            logfile = logger.setup(self.config_logfile, self.config_log_level)
-        if logfile != self.config_logfile:
-            log.warning("Log path %s not valid, falling back to default", self.config_logfile)
+            logfile = logger.setup(cli_param.logpath or self.config_logfile, self.config_log_level)
+        if logfile != os.path.abspath(self.config_logfile):
+            if logfile != os.path.abspath(cli_param.logpath):
+                log.warning("Log path %s not valid, falling back to default", self.config_logfile)
             self.config_logfile = logfile
+            s.config_logfile = logfile
             self._session.merge(s)
             try:
                 self._session.commit()
@@ -386,6 +397,9 @@ class ConfigSQL(object):
         self.db_configured = False
         self.save()
 
+    def get_book_path(self):
+        return self.config_calibre_split_dir if self.config_calibre_split_dir else self.config_calibre_dir
+
     def store_calibre_uuid(self, calibre_db, Library_table):
         try:
             calibre_uuid = calibre_db.session.query(Library_table).one_or_none()
@@ -405,20 +419,14 @@ def _encrypt_fields(session, secret_key):
         session.query(exists().where(_Settings.mail_password_e)).scalar()
     except OperationalError:
         with session.bind.connect() as conn:
-            conn.execute("ALTER TABLE settings ADD column 'mail_password_e' String")
-            conn.execute("ALTER TABLE settings ADD column 'config_goodreads_api_secret_e' String")
-            conn.execute("ALTER TABLE settings ADD column 'config_ldap_serv_password_e' String")
+            conn.execute(text("ALTER TABLE settings ADD column 'mail_password_e' String"))
+            conn.execute(text("ALTER TABLE settings ADD column 'config_ldap_serv_password_e' String"))
         session.commit()
         crypter = Fernet(secret_key)
-        settings = session.query(_Settings.mail_password, _Settings.config_goodreads_api_secret,
-                                 _Settings.config_ldap_serv_password).first()
+        settings = session.query(_Settings.mail_password, _Settings.config_ldap_serv_password).first()
         if settings.mail_password:
             session.query(_Settings).update(
                 {_Settings.mail_password_e: crypter.encrypt(settings.mail_password.encode())})
-        if settings.config_goodreads_api_secret:
-            session.query(_Settings).update(
-                {_Settings.config_goodreads_api_secret_e:
-                     crypter.encrypt(settings.config_goodreads_api_secret.encode())})
         if settings.config_ldap_serv_password:
             session.query(_Settings).update(
                 {_Settings.config_ldap_serv_password_e:
@@ -466,17 +474,33 @@ def _migrate_table(session, orm_class, secret_key=None):
             session.rollback()
 
 
-def autodetect_calibre_binary():
+def autodetect_calibre_binaries():
     if sys.platform == "win32":
-        calibre_path = ["C:\\program files\\calibre\\ebook-convert.exe",
-                        "C:\\program files(x86)\\calibre\\ebook-convert.exe",
-                        "C:\\program files(x86)\\calibre2\\ebook-convert.exe",
-                        "C:\\program files\\calibre2\\ebook-convert.exe"]
+        calibre_path = ["C:\\program files\\calibre\\",
+                        "C:\\program files(x86)\\calibre\\",
+                        "C:\\program files(x86)\\calibre2\\",
+                        "C:\\program files\\calibre2\\"]
     else:
-        calibre_path = ["/opt/calibre/ebook-convert"]
+        calibre_path = ["/opt/calibre/"]
     for element in calibre_path:
-        if os.path.isfile(element) and os.access(element, os.X_OK):
-            return element
+        supported_binary_paths = [os.path.join(element, binary) for binary in constants.SUPPORTED_CALIBRE_BINARIES.values()]
+        if all(os.path.isfile(binary_path) and os.access(binary_path, os.X_OK) for binary_path in supported_binary_paths):
+            values = [process_wait([binary_path, "--version"],
+                                   pattern=r'\(calibre (.*)\)') for binary_path in supported_binary_paths]
+            if all(values):
+                version = values[0].group(1)
+                log.debug("calibre version %s", version)
+                return element 
+    return ""
+
+
+def autodetect_converter_binary(calibre_path):
+    if sys.platform == "win32":
+        converter_path = os.path.join(calibre_path, "ebook-convert.exe")
+    else:
+        converter_path = os.path.join(calibre_path, "ebook-convert")
+    if calibre_path and os.path.isfile(converter_path) and os.access(converter_path, os.X_OK):
+        return converter_path
     return ""
 
 
@@ -531,7 +555,7 @@ def get_encryption_key(key_path):
     key_file = os.path.join(key_path, ".key")
     generate = True
     error = ""
-    if os.path.exists(key_file) and  os.path.getsize(key_file) > 32:
+    if os.path.exists(key_file) and os.path.getsize(key_file) > 32:
         with open(key_file, "rb") as f:
             key = f.read()
         try:

cps/constants.py

@@ -34,6 +34,8 @@ UPDATER_AVAILABLE = True
 
 # Base dir is parent of current file, necessary if called from different folder
 BASE_DIR            = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir))
+# if executable file the files should be placed in the parent dir (parallel to the exe file)
+
 STATIC_DIR          = os.path.join(BASE_DIR, 'cps', 'static')
 TEMPLATES_DIR       = os.path.join(BASE_DIR, 'cps', 'templates')
 TRANSLATIONS_DIR    = os.path.join(BASE_DIR, 'cps', 'translations')
@@ -49,6 +51,9 @@ if HOME_CONFIG:
     CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', home_dir)
 else:
     CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', BASE_DIR)
+    if getattr(sys, 'frozen', False):
+        CONFIG_DIR = os.path.abspath(os.path.join(CONFIG_DIR, os.pardir))
+
 
 DEFAULT_SETTINGS_FILE = "app.db"
 DEFAULT_GDRIVE_FILE = "gdrive.db"
@@ -144,13 +149,18 @@ del env_CALIBRE_PORT
 
 EXTENSIONS_AUDIO = {'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'}
 EXTENSIONS_CONVERT_FROM = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf',
-                           'txt', 'htmlz', 'rtf', 'odt', 'cbz', 'cbr']
+                           'txt', 'htmlz', 'rtf', 'odt', 'cbz', 'cbr', 'prc']
 EXTENSIONS_CONVERT_TO = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2',
                          'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt']
-EXTENSIONS_UPLOAD = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'djvu',
+EXTENSIONS_UPLOAD = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'cb7', 'djvu', 'djv',
                      'prc', 'doc', 'docx', 'fb2', 'html', 'rtf', 'lit', 'odt', 'mp3', 'mp4', 'ogg',
                      'opus', 'wav', 'flac', 'm4a', 'm4b'}
 
+_extension = ""
+if sys.platform == "win32":
+    _extension = ".exe"
+SUPPORTED_CALIBRE_BINARIES = {binary:binary + _extension for binary in ["ebook-convert", "calibredb"]}
+
 
 def has_flag(value, bit_flag):
     return bit_flag == (bit_flag & (value or 0))
@@ -163,13 +173,12 @@ def selected_roles(dictionary):
 BookMeta = namedtuple('BookMeta', 'file_path, extension, title, author, cover, description, tags, series, '
                                   'series_id, languages, publisher, pubdate, identifiers')
 
-STABLE_VERSION = {'version': '0.6.20'}
+# python build process likes to have x.y.zbw -> b for beta and w a counting number
+STABLE_VERSION = {'version': '0.6.22b'}
 
 NIGHTLY_VERSION = dict()
 NIGHTLY_VERSION[0] = '$Format:%H$'
 NIGHTLY_VERSION[1] = '$Format:%cI$'
-# NIGHTLY_VERSION[0] = 'bb7d2c6273ae4560e83950d36d64533343623a57'
-# NIGHTLY_VERSION[1] = '2018-09-09T10:13:08+02:00'
 
 # CACHE
 CACHE_TYPE_THUMBNAILS    = 'thumbnails'

cps/db.py

@@ -173,6 +173,9 @@ class Identifiers(Base):
             return "https://www.databazeknih.cz/knihy/{0}".format(self.val)
         elif self.val.lower().startswith("javascript:"):
             return quote(self.val)
+        elif self.val.lower().startswith("data:"):
+            link , __, __ = str.partition(self.val, ",")
+            return link
         else:
             return "{0}".format(self.val)
 
@@ -207,6 +210,9 @@ class Tags(Base):
     def get(self):
         return self.name
 
+    def __eq__(self, other):
+        return self.name == other
+
     def __repr__(self):
         return "<Tags('{0})>".format(self.name)
 
@@ -219,7 +225,7 @@ class Authors(Base):
     sort = Column(String(collation='NOCASE'))
     link = Column(String, nullable=False, default="")
 
-    def __init__(self, name, sort, link):
+    def __init__(self, name, sort, link=""):
         self.name = name
         self.sort = sort
         self.link = link
@@ -227,6 +233,9 @@ class Authors(Base):
     def get(self):
         return self.name
 
+    def __eq__(self, other):
+        return self.name == other
+
     def __repr__(self):
         return "<Authors('{0},{1}{2}')>".format(self.name, self.sort, self.link)
 
@@ -245,6 +254,9 @@ class Series(Base):
     def get(self):
         return self.name
 
+    def __eq__(self, other):
+        return self.name == other
+
     def __repr__(self):
         return "<Series('{0},{1}')>".format(self.name, self.sort)
 
@@ -261,6 +273,9 @@ class Ratings(Base):
     def get(self):
         return self.rating
 
+    def __eq__(self, other):
+        return self.rating == other
+
     def __repr__(self):
         return "<Ratings('{0}')>".format(self.rating)
 
@@ -275,11 +290,14 @@ class Languages(Base):
         self.lang_code = lang_code
 
     def get(self):
-        if self.language_name:
+        if hasattr(self, "language_name"):
             return self.language_name
         else:
             return self.lang_code
 
+    def __eq__(self, other):
+        return self.lang_code == other
+
     def __repr__(self):
         return "<Languages('{0}')>".format(self.lang_code)
 
@@ -298,6 +316,9 @@ class Publishers(Base):
     def get(self):
         return self.name
 
+    def __eq__(self, other):
+        return self.name == other
+
     def __repr__(self):
         return "<Publishers('{0},{1}')>".format(self.name, self.sort)
 
@@ -642,7 +663,7 @@ class CalibreDB:
 
         cls.session_factory = scoped_session(sessionmaker(autocommit=False,
                                                           autoflush=True,
-                                                          bind=cls.engine))
+                                                          bind=cls.engine, future=True))
         for inst in cls.instances:
             inst.init_session()
 
@@ -818,8 +839,7 @@ class CalibreDB:
         entries = list()
         pagination = list()
         try:
-            pagination = Pagination(page, pagesize,
-                                    len(query.all()))
+            pagination = Pagination(page, pagesize, query.count())
             entries = query.order_by(*order).offset(off).limit(pagesize).all()
         except Exception as ex:
             log.error_or_exception(ex)
@@ -829,8 +849,6 @@ class CalibreDB:
 
     # Orders all Authors in the list according to authors sort
     def order_authors(self, entries, list_return=False, combined=False):
-        # entries_copy = copy.deepcopy(entries)
-        # entries_copy =[]
         for entry in entries:
             if combined:
                 sort_authors = entry.Books.author_sort.split('&')
@@ -995,7 +1013,12 @@ class CalibreDB:
                 title = title[len(prep):] + ', ' + prep
             return title.strip()
 
-        conn = conn or self.session.connection().connection.connection
+        try:
+            # sqlalchemy <1.4.24
+            conn = conn or self.session.connection().connection.driver_connection
+        except AttributeError:
+            # sqlalchemy >1.4.24 and sqlalchemy 2.0
+            conn = conn or self.session.connection().connection.connection
         try:
             conn.create_function("title_sort", 1, _title_sort)
         except sqliteOperationalError:

cps/dep_check.py

@@ -61,7 +61,7 @@ def dependency_check(optional=False):
     deps = load_dependencies(optional)
     for dep in deps:
         try:
-            dep_version_int = [int(x) for x in dep[0].split('.')]
+            dep_version_int = [int(x) if x.isnumeric() else 0 for x in dep[0].split('.')]
             low_check = [int(x) for x in dep[3].split('.')]
             high_check = [int(x) for x in dep[5].split('.')]
         except AttributeError:

cps/editbooks.py

@@ -25,29 +25,43 @@ from datetime import datetime
 import json
 from shutil import copyfile
 from uuid import uuid4
-from markupsafe import escape  # dependency of flask
+from markupsafe import escape, Markup  # dependency of flask
 from functools import wraps
+# from lxml.etree import ParserError
 
-try:
-    from lxml.html.clean import clean_html
-except ImportError:
-    clean_html = None
+#try:
+#    # at least bleach 6.0 is needed -> incomplatible change from list arguments to set arguments
+#    from bleach import clean_text as clean_html
+#    BLEACH = True
+#except ImportError:
+#    try:
+#        BLEACH = False
+#        from nh3 import clean as clean_html
+#    except ImportError:
+#        try:
+#            BLEACH = False
+#            from lxml.html.clean import clean_html
+#        except ImportError:
+#            clean_html = None
 
-from flask import Blueprint, request, flash, redirect, url_for, abort, Markup, Response
+from flask import Blueprint, request, flash, redirect, url_for, abort, Response
 from flask_babel import gettext as _
 from flask_babel import lazy_gettext as N_
 from flask_babel import get_locale
 from flask_login import current_user, login_required
 from sqlalchemy.exc import OperationalError, IntegrityError, InterfaceError
 from sqlalchemy.orm.exc import StaleDataError
+from sqlalchemy.sql.expression import func
 
 from . import constants, logger, isoLanguages, gdriveutils, uploader, helper, kobo_sync_status
+from .clean_html import clean_string
 from . import config, ub, db, calibre_db
 from .services.worker import WorkerThread
 from .tasks.upload import TaskUpload
 from .render_template import render_title_template
 from .usermanagement import login_required_if_no_ano
 from .kobo_sync_status import change_archived_books
+from .redirect import get_redirect_location
 
 
 editbook = Blueprint('edit-book', __name__)
@@ -84,7 +98,7 @@ def delete_book_from_details(book_id):
 @editbook.route("/delete/<int:book_id>/<string:book_format>", methods=["POST"])
 @login_required
 def delete_book_ajax(book_id, book_format):
-    return delete_book_from_table(book_id, book_format, False)
+    return delete_book_from_table(book_id, book_format, False, request.form.to_dict().get('location', ""))
 
 
 @editbook.route("/admin/book/<int:book_id>", methods=['GET'])
@@ -125,7 +139,7 @@ def edit_book(book_id):
             edited_books_id = book.id
             modify_date = True
             title_author_error = helper.update_dir_structure(edited_books_id,
-                                                             config.config_calibre_dir,
+                                                             config.get_book_path(),
                                                              input_authors[0],
                                                              renamed_author=renamed)
         if title_author_error:
@@ -270,7 +284,7 @@ def upload():
                                                   meta.extension.lower())
                 else:
                     error = helper.update_dir_structure(book_id,
-                                                        config.config_calibre_dir,
+                                                        config.get_book_path(),
                                                         input_authors[0],
                                                         meta.file_path,
                                                         title_dir + meta.extension.lower(),
@@ -320,7 +334,7 @@ def convert_bookformat(book_id):
         return redirect(url_for('edit-book.show_edit_book', book_id=book_id))
 
     log.info('converting: book id: %s from: %s to: %s', book_id, book_format_from, book_format_to)
-    rtn = helper.convert_book_format(book_id, config.config_calibre_dir, book_format_from.upper(),
+    rtn = helper.convert_book_format(book_id, config.get_book_path(), book_format_from.upper(),
                                      book_format_to.upper(), current_user.name)
 
     if rtn is None:
@@ -390,7 +404,7 @@ def edit_list_book(param):
         elif param == 'title':
             sort_param = book.sort
             if handle_title_on_edit(book, vals.get('value', "")):
-                rename_error = helper.update_dir_structure(book.id, config.config_calibre_dir)
+                rename_error = helper.update_dir_structure(book.id, config.get_book_path())
                 if not rename_error:
                     ret = Response(json.dumps({'success': True, 'newValue':  book.title}),
                                    mimetype='application/json')
@@ -408,7 +422,7 @@ def edit_list_book(param):
                            mimetype='application/json')
         elif param == 'authors':
             input_authors, __, renamed = handle_author_on_edit(book, vals['value'], vals.get('checkA', None) == "true")
-            rename_error = helper.update_dir_structure(book.id, config.config_calibre_dir, input_authors[0],
+            rename_error = helper.update_dir_structure(book.id, config.get_book_path(), input_authors[0],
                                                        renamed_author=renamed)
             if not rename_error:
                 ret = Response(json.dumps({
@@ -470,7 +484,7 @@ def get_sorted_entry(field, bookid):
             if field == 'sort':
                 return json.dumps({'sort': book.title})
             if field == 'author_sort':
-                return json.dumps({'author_sort': book.author})
+                return json.dumps({'authors': " & ".join([a.name for a in calibre_db.order_authors([book])])})
     return ""
 
 
@@ -512,10 +526,10 @@ def merge_list_book():
                     for element in from_book.data:
                         if element.format not in to_file:
                             # create new data entry with: book_id, book_format, uncompressed_size, name
-                            filepath_new = os.path.normpath(os.path.join(config.config_calibre_dir,
+                            filepath_new = os.path.normpath(os.path.join(config.get_book_path(),
                                                                          to_book.path,
                                                                          to_name + "." + element.format.lower()))
-                            filepath_old = os.path.normpath(os.path.join(config.config_calibre_dir,
+                            filepath_old = os.path.normpath(os.path.join(config.get_book_path(),
                                                                          from_book.path,
                                                                          element.name + "." + element.format.lower()))
                             copyfile(filepath_old, filepath_new)
@@ -555,7 +569,7 @@ def table_xchange_author_title():
 
             if edited_books_id:
                 # toDo: Handle error
-                edit_error = helper.update_dir_structure(edited_books_id, config.config_calibre_dir, input_authors[0],
+                edit_error = helper.update_dir_structure(edited_books_id, config.get_book_path(), input_authors[0],
                                                          renamed_author=renamed)
             if modify_date:
                 book.last_modified = datetime.utcnow()
@@ -598,6 +612,8 @@ def identifier_list(to_save, book):
         val_key = id_val_prefix + type_key[len(id_type_prefix):]
         if val_key not in to_save.keys():
             continue
+        if to_save[val_key].startswith("data:"):
+            to_save[val_key], __, __ = str.partition(to_save[val_key], ",")
         result.append(db.Identifiers(to_save[val_key], type_value, book.id))
     return result
 
@@ -750,7 +766,7 @@ def move_coverfile(meta, db_book):
         cover_file = meta.cover
     else:
         cover_file = os.path.join(constants.STATIC_DIR, 'generic_cover.jpg')
-    new_cover_path = os.path.join(config.config_calibre_dir, db_book.path)
+    new_cover_path = os.path.join(config.get_book_path(), db_book.path)
     try:
         os.makedirs(new_cover_path, exist_ok=True)
         copyfile(cover_file, os.path.join(new_cover_path, "cover.jpg"))
@@ -809,7 +825,7 @@ def delete_whole_book(book_id, book):
     calibre_db.session.query(db.Books).filter(db.Books.id == book_id).delete()
 
 
-def render_delete_book_result(book_format, json_response, warning, book_id):
+def render_delete_book_result(book_format, json_response, warning, book_id, location=""):
     if book_format:
         if json_response:
             return json.dumps([warning, {"location": url_for("edit-book.show_edit_book", book_id=book_id),
@@ -821,22 +837,22 @@ def render_delete_book_result(book_format, json_response, warning, book_id):
             return redirect(url_for('edit-book.show_edit_book', book_id=book_id))
     else:
         if json_response:
-            return json.dumps([warning, {"location": url_for('web.index'),
+            return json.dumps([warning, {"location": get_redirect_location(location, "web.index"),
                                          "type": "success",
                                          "format": book_format,
                                          "message": _('Book Successfully Deleted')}])
         else:
             flash(_('Book Successfully Deleted'), category="success")
-            return redirect(url_for('web.index'))
+            return redirect(get_redirect_location(location, "web.index"))
 
 
-def delete_book_from_table(book_id, book_format, json_response):
+def delete_book_from_table(book_id, book_format, json_response, location=""):
     warning = {}
     if current_user.role_delete_books():
         book = calibre_db.get_book(book_id)
         if book:
             try:
-                result, error = helper.delete_book(book, config.config_calibre_dir, book_format=book_format.upper())
+                result, error = helper.delete_book(book, config.get_book_path(), book_format=book_format.upper())
                 if not result:
                     if json_response:
                         return json.dumps([{"location": url_for("edit-book.show_edit_book", book_id=book_id),
@@ -877,7 +893,7 @@ def delete_book_from_table(book_id, book_format, json_response):
         else:
             # book not found
             log.error('Book with id "%s" could not be deleted: not found', book_id)
-        return render_delete_book_result(book_format, json_response, warning, book_id)
+        return render_delete_book_result(book_format, json_response, warning, book_id, location)
     message = _("You are missing permissions to delete books")
     if json_response:
         return json.dumps({"location": url_for("edit-book.show_edit_book", book_id=book_id),
@@ -989,7 +1005,18 @@ def edit_book_series_index(series_index, book):
 def edit_book_comments(comments, book):
     modify_date = False
     if comments:
-        comments = clean_html(comments)
+        comments = clean_string(comments, book.id)
+        #try:
+        #    if BLEACH:
+        #        comments = clean_html(comments, tags=set(), attributes=set())
+        #    else:
+        #        comments = clean_html(comments)
+        #except ParserError as e:
+        #    log.error("Comments of book {} are corrupted: {}".format(book.id, e))
+         #   comments = ""
+        #except TypeError as e:
+        #    log.error("Comments can't be parsed, maybe 'lxml' is too new, try installing 'bleach': {}".format(e))
+        #    comments = ""
     if len(book.comments):
         if book.comments[0].text != comments:
             book.comments[0].text = comments
@@ -1047,7 +1074,19 @@ def edit_cc_data_value(book_id, book, c, to_save, cc_db_value, cc_string):
     elif c.datatype == 'comments':
         to_save[cc_string] = Markup(to_save[cc_string]).unescape()
         if to_save[cc_string]:
-            to_save[cc_string] = clean_html(to_save[cc_string])
+            to_save[cc_string] = clean_string(to_save[cc_string], book_id)
+            #try:
+            #    if BLEACH:
+            #        to_save[cc_string] = clean_html(to_save[cc_string], tags=set(), attributes=set())
+            #    else:
+             #       to_save[cc_string] = clean_html(to_save[cc_string])
+            #except ParserError as e:
+            #    log.error("Customs Comments of book {} are corrupted: {}".format(book_id, e))
+            #    to_save[cc_string] = ""
+            #except TypeError as e:
+            #    to_save[cc_string] = ""
+            #    log.error("Customs Comments can't be parsed, maybe 'lxml' is too new, "
+            #              "try installing 'bleach': {}".format(e))
     elif c.datatype == 'datetime':
         try:
             to_save[cc_string] = datetime.strptime(to_save[cc_string], "%Y-%m-%d")
@@ -1169,7 +1208,7 @@ def upload_single_file(file_request, book, book_id):
                 return False
 
             file_name = book.path.rsplit('/', 1)[-1]
-            filepath = os.path.normpath(os.path.join(config.config_calibre_dir, book.path))
+            filepath = os.path.normpath(os.path.join(config.get_book_path(), book.path))
             saved_filename = os.path.join(filepath, file_name + '.' + file_ext)
 
             # check if file path exists, otherwise create it, copy file to calibre path and delete temp file
@@ -1211,7 +1250,7 @@ def upload_single_file(file_request, book, book_id):
 
             return uploader.process(
                 saved_filename, *os.path.splitext(requested_file.filename),
-                rarExecutable=config.config_rarfile_location)
+                rar_executable=config.config_rarfile_location)
     return None
 
 
@@ -1245,18 +1284,18 @@ def handle_title_on_edit(book, book_title):
 
 
 def handle_author_on_edit(book, author_name, update_stored=True):
+    change = False
     # handle author(s)
     input_authors, renamed = prepare_authors(author_name)
 
-    change = modify_database_object(input_authors, book.authors, db.Authors, calibre_db.session, 'author')
-
+    # change |= modify_database_object(input_authors, book.authors, db.Authors, calibre_db.session, 'author')
     # Search for each author if author is in database, if not, author name and sorted author name is generated new
     # everything then is assembled for sorted author field in database
     sort_authors_list = list()
     for inp in input_authors:
         stored_author = calibre_db.session.query(db.Authors).filter(db.Authors.name == inp).first()
         if not stored_author:
-            stored_author = helper.get_sorted_author(inp)
+            stored_author = helper.get_sorted_author(inp.replace('|', ','))
         else:
             stored_author = stored_author.sort
         sort_authors_list.append(helper.get_sorted_author(stored_author))
@@ -1264,6 +1303,9 @@ def handle_author_on_edit(book, author_name, update_stored=True):
     if book.author_sort != sort_authors and update_stored:
         book.author_sort = sort_authors
         change = True
+
+    change |= modify_database_object(input_authors, book.authors, db.Authors, calibre_db.session, 'author')
+
     return input_authors, change, renamed
 
 
@@ -1271,14 +1313,15 @@ def search_objects_remove(db_book_object, db_type, input_elements):
     del_elements = []
     for c_elements in db_book_object:
         found = False
-        if db_type == 'languages':
-            type_elements = c_elements.lang_code
-        elif db_type == 'custom':
+        #if db_type == 'languages':
+        #    type_elements = c_elements.lang_code
+        if db_type == 'custom':
             type_elements = c_elements.value
         else:
-            type_elements = c_elements.name
+            # type_elements = c_elements.name
+            type_elements = c_elements
         for inp_element in input_elements:
-            if inp_element.lower() == type_elements.lower():
+            if type_elements == inp_element:
                 found = True
                 break
         # if the element was not found in the new list, add it to remove list
@@ -1292,13 +1335,11 @@ def search_objects_add(db_book_object, db_type, input_elements):
     for inp_element in input_elements:
         found = False
         for c_elements in db_book_object:
-            if db_type == 'languages':
-                type_elements = c_elements.lang_code
-            elif db_type == 'custom':
+            if db_type == 'custom':
                 type_elements = c_elements.value
             else:
-                type_elements = c_elements.name
-            if inp_element == type_elements:
+                type_elements = c_elements
+            if type_elements == inp_element:
                 found = True
                 break
         if not found:
@@ -1314,6 +1355,7 @@ def remove_objects(db_book_object, db_session, del_elements):
             changed = True
             if len(del_element.books) == 0:
                 db_session.delete(del_element)
+                db_session.flush()
     return changed
 
 
@@ -1327,27 +1369,34 @@ def add_objects(db_book_object, db_object, db_session, db_type, add_elements):
         db_filter = db_object.name
     for add_element in add_elements:
         # check if an element with that name exists
-        db_element = db_session.query(db_object).filter(db_filter == add_element).first()
+        changed = True
+        # db_session.query(db.Tags).filter((func.lower(db.Tags.name).ilike("GênOt"))).all()
+        db_element = db_session.query(db_object).filter((func.lower(db_filter).ilike(add_element))).first()
+        # db_element = db_session.query(db_object).filter(func.lower(db_filter) == add_element.lower()).first()
         # if no element is found add it
-        if db_type == 'author':
-            new_element = db_object(add_element, helper.get_sorted_author(add_element.replace('|', ',')), "")
-        elif db_type == 'series':
-            new_element = db_object(add_element, add_element)
-        elif db_type == 'custom':
-            new_element = db_object(value=add_element)
-        elif db_type == 'publisher':
-            new_element = db_object(add_element, None)
-        else:  # db_type should be tag or language
-            new_element = db_object(add_element)
         if db_element is None:
-            changed = True
+            if db_type == 'author':
+                new_element = db_object(add_element, helper.get_sorted_author(add_element.replace('|', ',')))
+            elif db_type == 'series':
+                new_element = db_object(add_element, add_element)
+            elif db_type == 'custom':
+                new_element = db_object(value=add_element)
+            elif db_type == 'publisher':
+                new_element = db_object(add_element, None)
+            else:  # db_type should be tag or language
+                new_element = db_object(add_element)
             db_session.add(new_element)
             db_book_object.append(new_element)
         else:
-            db_element = create_objects_for_addition(db_element, add_element, db_type)
+            db_no_case = db_session.query(db_object).filter(db_filter == add_element).first()
+            if db_no_case:
+                # check for new case of element
+                db_element = create_objects_for_addition(db_element, add_element, db_type)
+            else:
+                db_element = create_objects_for_addition(db_element, add_element, db_type)
             # add element to book
-            changed = True
             db_book_object.append(db_element)
+
     return changed
 
 
@@ -1382,13 +1431,24 @@ def modify_database_object(input_elements, db_book_object, db_object, db_session
     if not isinstance(input_elements, list):
         raise TypeError(str(input_elements) + " should be passed as a list")
     input_elements = [x for x in input_elements if x != '']
-    # we have all input element (authors, series, tags) names now
+
+    changed = False
+    # If elements are renamed (upper lower case), rename it
+    for rec_a, rec_b in zip(db_book_object, input_elements):
+        if db_type == "custom":
+            if rec_a.value.casefold() == rec_b.casefold() and rec_a.value != rec_b:
+                create_objects_for_addition(rec_a, rec_b, db_type)
+        else:
+            if rec_a.get().casefold() == rec_b.casefold() and rec_a.get() != rec_b:
+                create_objects_for_addition(rec_a, rec_b, db_type)
+        # we have all input element (authors, series, tags) names now
     # 1. search for elements to remove
     del_elements = search_objects_remove(db_book_object, db_type, input_elements)
     # 2. search for elements that need to be added
     add_elements = search_objects_add(db_book_object, db_type, input_elements)
+
     # if there are elements to remove, we remove them now
-    changed = remove_objects(db_book_object, db_session, del_elements)
+    changed |= remove_objects(db_book_object, db_session, del_elements)
     # if there are elements to add, we add them now!
     if len(add_elements) > 0:
         changed |= add_objects(db_book_object, db_object, db_session, db_type, add_elements)

cps/embed_helper.py

@@ -0,0 +1,63 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2024 OzzieIsaacs
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+from uuid import uuid4
+import os
+
+from .file_helper import get_temp_dir
+from .subproc_wrapper import process_open
+from . import logger, config
+from .constants import SUPPORTED_CALIBRE_BINARIES
+
+log = logger.create()
+
+
+def do_calibre_export(book_id, book_format):
+    try:
+        quotes = [3, 5, 7, 9]
+        tmp_dir = get_temp_dir()
+        calibredb_binarypath = get_calibre_binarypath("calibredb")
+        temp_file_name = str(uuid4())
+        my_env = os.environ.copy()
+        if config.config_calibre_split:
+            my_env['CALIBRE_OVERRIDE_DATABASE_PATH'] = os.path.join(config.config_calibre_dir, "metadata.db")
+            library_path = config.config_calibre_split_dir
+        else:
+            library_path = config.config_calibre_dir
+        opf_command = [calibredb_binarypath, 'export', '--dont-write-opf', '--with-library', library_path,
+                       '--to-dir', tmp_dir, '--formats', book_format, "--template", "{}".format(temp_file_name),
+                       str(book_id)]
+        p = process_open(opf_command, quotes, my_env)
+        _, err = p.communicate()
+        if err:
+            log.error('Metadata embedder encountered an error: %s', err)
+        return tmp_dir, temp_file_name
+    except OSError as ex:
+        # ToDo real error handling
+        log.error_or_exception(ex)
+        return None, None
+
+
+def get_calibre_binarypath(binary):
+    binariesdir = config.config_binariesdir
+    if binariesdir:
+        try:
+            return os.path.join(binariesdir, SUPPORTED_CALIBRE_BINARIES[binary])
+        except KeyError as ex:
+            log.error("Binary not supported by Calibre-Web: %s", SUPPORTED_CALIBRE_BINARIES[binary])
+            pass
+    return ""

cps/epub.py

@@ -21,10 +21,13 @@ import zipfile
 from lxml import etree
 
 from . import isoLanguages, cover
-from . import config
+from . import config, logger
 from .helper import split_authors
+from .epub_helper import get_content_opf, default_ns
 from .constants import BookMeta
 
+log = logger.create()
+
 
 def _extract_cover(zip_file, cover_file, cover_path, tmp_file_name):
     if cover_file is None:
@@ -43,21 +46,17 @@ def _extract_cover(zip_file, cover_file, cover_path, tmp_file_name):
     return cover.cover_processing(tmp_file_name, cf, extension)
 
 def get_epub_layout(book, book_data):
-    ns = {
-        'n': 'urn:oasis:names:tc:opendocument:xmlns:container',
-        'pkg': 'http://www.idpf.org/2007/opf',
-    }
-    file_path = os.path.normpath(os.path.join(config.config_calibre_dir, book.path, book_data.name + "." + book_data.format.lower()))
+    file_path = os.path.normpath(os.path.join(config.get_book_path(),
+                                              book.path, book_data.name + "." + book_data.format.lower()))
 
-    epubZip = zipfile.ZipFile(file_path)
-    txt = epubZip.read('META-INF/container.xml')
-    tree = etree.fromstring(txt)
-    cfname = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0]
-    cf = epubZip.read(cfname)
-    tree = etree.fromstring(cf)
-    p = tree.xpath('/pkg:package/pkg:metadata', namespaces=ns)[0]
+    try:
+        tree, __ = get_content_opf(file_path, default_ns)
+        p = tree.xpath('/pkg:package/pkg:metadata', namespaces=default_ns)[0]
 
-    layout = p.xpath('pkg:meta[@property="rendition:layout"]/text()', namespaces=ns)
+        layout = p.xpath('pkg:meta[@property="rendition:layout"]/text()', namespaces=default_ns)
+    except (etree.XMLSyntaxError, KeyError, IndexError, OSError) as e:
+        log.error("Could not parse epub metadata of book {} during kobo sync: {}".format(book.id, e))
+        layout = []
 
     if len(layout) == 0:
         return None
@@ -72,13 +71,7 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
         'dc': 'http://purl.org/dc/elements/1.1/'
     }
 
-    epub_zip = zipfile.ZipFile(tmp_file_path)
-
-    txt = epub_zip.read('META-INF/container.xml')
-    tree = etree.fromstring(txt)
-    cf_name = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0]
-    cf = epub_zip.read(cf_name)
-    tree = etree.fromstring(cf)
+    tree, cf_name = get_content_opf(tmp_file_path, ns)
 
     cover_path = os.path.dirname(cf_name)
 
@@ -96,7 +89,7 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
             elif s == 'date':
                 epub_metadata[s] = tmp[0][:10]
             else:
-                epub_metadata[s] = tmp[0]
+                epub_metadata[s] = tmp[0].strip()
         else:
             epub_metadata[s] = 'Unknown'
 
@@ -121,6 +114,7 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
 
     epub_metadata = parse_epub_series(ns, tree, epub_metadata)
 
+    epub_zip = zipfile.ZipFile(tmp_file_path)
     cover_file = parse_epub_cover(ns, tree, epub_zip, cover_path, tmp_file_path)
 
     identifiers = []

cps/epub_helper.py

@@ -0,0 +1,166 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2018 lemmsh, Kennyl, Kyosfonica, matthazinski
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+import zipfile
+from lxml import etree
+
+from . import isoLanguages
+
+default_ns = {
+    'n': 'urn:oasis:names:tc:opendocument:xmlns:container',
+    'pkg': 'http://www.idpf.org/2007/opf',
+}
+
+OPF_NAMESPACE = "http://www.idpf.org/2007/opf"
+PURL_NAMESPACE = "http://purl.org/dc/elements/1.1/"
+
+OPF = "{%s}" % OPF_NAMESPACE
+PURL = "{%s}" % PURL_NAMESPACE
+
+etree.register_namespace("opf", OPF_NAMESPACE)
+etree.register_namespace("dc", PURL_NAMESPACE)
+
+OPF_NS = {None: OPF_NAMESPACE}  # the default namespace (no prefix)
+NSMAP = {'dc': PURL_NAMESPACE, 'opf': OPF_NAMESPACE}
+
+
+def updateEpub(src, dest, filename, data, ):
+    # create a temp copy of the archive without filename
+    with zipfile.ZipFile(src, 'r') as zin:
+        with zipfile.ZipFile(dest, 'w') as zout:
+            zout.comment = zin.comment # preserve the comment
+            for item in zin.infolist():
+                if item.filename != filename:
+                    zout.writestr(item, zin.read(item.filename))
+
+    # now add filename with its new data
+    with zipfile.ZipFile(dest, mode='a', compression=zipfile.ZIP_DEFLATED) as zf:
+        zf.writestr(filename, data)
+
+
+def get_content_opf(file_path, ns=default_ns):
+    epubZip = zipfile.ZipFile(file_path)
+    txt = epubZip.read('META-INF/container.xml')
+    tree = etree.fromstring(txt)
+    cf_name = tree.xpath('n:rootfiles/n:rootfile/@full-path', namespaces=ns)[0]
+    cf = epubZip.read(cf_name)
+
+    return etree.fromstring(cf), cf_name
+
+
+def create_new_metadata_backup(book,  custom_columns, export_language, translated_cover_name, lang_type=3):
+    # generate root package element
+    package = etree.Element(OPF + "package", nsmap=OPF_NS)
+    package.set("unique-identifier", "uuid_id")
+    package.set("version", "2.0")
+
+    # generate metadata element and all sub elements of it
+    metadata = etree.SubElement(package, "metadata", nsmap=NSMAP)
+    identifier = etree.SubElement(metadata, PURL + "identifier", id="calibre_id", nsmap=NSMAP)
+    identifier.set(OPF + "scheme", "calibre")
+    identifier.text = str(book.id)
+    identifier2 = etree.SubElement(metadata, PURL + "identifier", id="uuid_id", nsmap=NSMAP)
+    identifier2.set(OPF + "scheme", "uuid")
+    identifier2.text = book.uuid
+    for i in book.identifiers:
+        identifier = etree.SubElement(metadata, PURL + "identifier", nsmap=NSMAP)
+        identifier.set(OPF + "scheme", i.format_type())
+        identifier.text = str(i.val)
+    title = etree.SubElement(metadata, PURL + "title", nsmap=NSMAP)
+    title.text = book.title
+    for author in book.authors:
+        creator = etree.SubElement(metadata, PURL + "creator", nsmap=NSMAP)
+        creator.text = str(author.name)
+        creator.set(OPF + "file-as", book.author_sort)     # ToDo Check
+        creator.set(OPF + "role", "aut")
+    contributor = etree.SubElement(metadata, PURL + "contributor", nsmap=NSMAP)
+    contributor.text = "calibre (5.7.2) [https://calibre-ebook.com]"
+    contributor.set(OPF + "file-as", "calibre")     # ToDo Check
+    contributor.set(OPF + "role", "bkp")
+
+    date = etree.SubElement(metadata, PURL + "date", nsmap=NSMAP)
+    date.text = '{d.year:04}-{d.month:02}-{d.day:02}T{d.hour:02}:{d.minute:02}:{d.second:02}'.format(d=book.pubdate)
+    if book.comments and book.comments[0].text:
+        for b in book.comments:
+            description = etree.SubElement(metadata, PURL + "description", nsmap=NSMAP)
+            description.text = b.text
+    for b in book.publishers:
+        publisher = etree.SubElement(metadata, PURL + "publisher", nsmap=NSMAP)
+        publisher.text = str(b.name)
+    if not book.languages:
+        language = etree.SubElement(metadata, PURL + "language", nsmap=NSMAP)
+        language.text = export_language
+    else:
+        for b in book.languages:
+            language = etree.SubElement(metadata, PURL + "language", nsmap=NSMAP)
+            language.text = str(b.lang_code) if lang_type == 3 else isoLanguages.get(part3=b.lang_code).part1
+    for b in book.tags:
+        subject = etree.SubElement(metadata, PURL + "subject", nsmap=NSMAP)
+        subject.text = str(b.name)
+    etree.SubElement(metadata, "meta", name="calibre:author_link_map",
+                     content="{" + ", ".join(['"' + str(a.name) + '": ""' for a in book.authors]) + "}",
+                     nsmap=NSMAP)
+    for b in book.series:
+        etree.SubElement(metadata, "meta", name="calibre:series",
+                         content=str(str(b.name)),
+                         nsmap=NSMAP)
+    if book.series:
+        etree.SubElement(metadata, "meta", name="calibre:series_index",
+                         content=str(book.series_index),
+                         nsmap=NSMAP)
+    if len(book.ratings) and book.ratings[0].rating > 0:
+        etree.SubElement(metadata, "meta", name="calibre:rating",
+                         content=str(book.ratings[0].rating),
+                         nsmap=NSMAP)
+    etree.SubElement(metadata, "meta", name="calibre:timestamp",
+                     content='{d.year:04}-{d.month:02}-{d.day:02}T{d.hour:02}:{d.minute:02}:{d.second:02}'.format(
+                         d=book.timestamp),
+                     nsmap=NSMAP)
+    etree.SubElement(metadata, "meta", name="calibre:title_sort",
+                     content=book.sort,
+                     nsmap=NSMAP)
+    sequence = 0
+    for cc in custom_columns:
+        value = None
+        extra = None
+        cc_entry = getattr(book, "custom_column_" + str(cc.id))
+        if cc_entry.__len__():
+            value = [c.value for c in cc_entry] if cc.is_multiple else cc_entry[0].value
+            extra = cc_entry[0].extra if hasattr(cc_entry[0], "extra") else None
+        etree.SubElement(metadata, "meta", name="calibre:user_metadata:#{}".format(cc.label),
+                         content=cc.to_json(value, extra, sequence),
+                         nsmap=NSMAP)
+        sequence += 1
+
+    # generate guide element and all sub elements of it
+    # Title is translated from default export language
+    guide = etree.SubElement(package, "guide")
+    etree.SubElement(guide, "reference", type="cover", title=translated_cover_name, href="cover.jpg")
+
+    return package
+
+def replace_metadata(tree, package):
+    rep_element = tree.xpath('/pkg:package/pkg:metadata', namespaces=default_ns)[0]
+    new_element = package.xpath('//metadata', namespaces=default_ns)[0]
+    tree.replace(rep_element, new_element)
+    return etree.tostring(tree,
+                   xml_declaration=True,
+                   encoding='utf-8',
+                   pretty_print=True).decode('utf-8')
+
+

cps/file_helper.py

@@ -0,0 +1,32 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2023 OzzieIsaacs
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from tempfile import gettempdir
+import os
+import shutil
+
+def get_temp_dir():
+    tmp_dir = os.path.join(gettempdir(), 'calibre_web')
+    if not os.path.isdir(tmp_dir):
+        os.mkdir(tmp_dir)
+    return tmp_dir
+
+
+def del_temp_dir():
+    tmp_dir = os.path.join(gettempdir(), 'calibre_web')
+    shutil.rmtree(tmp_dir)

cps/gdrive.py

@@ -23,7 +23,6 @@
 import os
 import hashlib
 import json
-import tempfile
 from uuid import uuid4
 from time import time
 from shutil import move, copyfile
@@ -34,6 +33,7 @@ from flask_login import login_required
 
 from . import logger, gdriveutils, config, ub, calibre_db, csrf
 from .admin import admin_required
+from .file_helper import get_temp_dir
 
 gdrive = Blueprint('gdrive', __name__, url_prefix='/gdrive')
 log = logger.create()
@@ -139,9 +139,7 @@ try:
                 dbpath = os.path.join(config.config_calibre_dir, "metadata.db").encode()
                 if not response['deleted'] and response['file']['title'] == 'metadata.db' \
                     and response['file']['md5Checksum'] != hashlib.md5(dbpath):  # nosec
-                    tmp_dir = os.path.join(tempfile.gettempdir(), 'calibre_web')
-                    if not os.path.isdir(tmp_dir):
-                        os.mkdir(tmp_dir)
+                    tmp_dir = get_temp_dir()
 
                     log.info('Database file updated')
                     copyfile(dbpath, os.path.join(tmp_dir, "metadata.db_" + str(current_milli_time())))

cps/gdriveutils.py

@@ -34,7 +34,6 @@ except ImportError:
     from sqlalchemy.ext.declarative import declarative_base
 from sqlalchemy.exc import OperationalError, InvalidRequestError, IntegrityError
 from sqlalchemy.orm.exc import StaleDataError
-from sqlalchemy.sql.expression import text
 
 try:
     from httplib2 import __version__ as httplib2_version
@@ -147,7 +146,7 @@ engine = create_engine('sqlite:///{0}'.format(cli_param.gd_path), echo=False)
 Base = declarative_base()
 
 # Open session for database connection
-Session = sessionmaker()
+Session = sessionmaker(autoflush=False)
 Session.configure(bind=engine)
 session = scoped_session(Session)
 

cps/helper.py

@@ -22,12 +22,13 @@ import random
 import io
 import mimetypes
 import re
+import regex
 import shutil
 import socket
 from datetime import datetime, timedelta
-from tempfile import gettempdir
 import requests
 import unidecode
+from uuid import uuid4
 
 from flask import send_from_directory, make_response, redirect, abort, url_for
 from flask_babel import gettext as _
@@ -54,12 +55,16 @@ from . import calibre_db, cli_param
 from .tasks.convert import TaskConvert
 from . import logger, config, db, ub, fs
 from . import gdriveutils as gd
-from .constants import STATIC_DIR as _STATIC_DIR, CACHE_TYPE_THUMBNAILS, THUMBNAIL_TYPE_COVER, THUMBNAIL_TYPE_SERIES
+from .constants import (STATIC_DIR as _STATIC_DIR, CACHE_TYPE_THUMBNAILS, THUMBNAIL_TYPE_COVER, THUMBNAIL_TYPE_SERIES,
+                        SUPPORTED_CALIBRE_BINARIES)
 from .subproc_wrapper import process_wait
 from .services.worker import WorkerThread
 from .tasks.mail import TaskEmail
 from .tasks.thumbnail import TaskClearCoverThumbnailCache, TaskGenerateCoverThumbnails
 from .tasks.metadata_backup import TaskBackupMetadata
+from .file_helper import get_temp_dir
+from .epub_helper import get_content_opf, create_new_metadata_backup, updateEpub, replace_metadata
+from .embed_helper import do_calibre_export
 
 log = logger.create()
 
@@ -191,7 +196,7 @@ def check_send_to_ereader(entry):
 # Check if a reader is existing for any of the book formats, if not, return empty list, otherwise return
 # list with supported formats
 def check_read_formats(entry):
-    extensions_reader = {'TXT', 'PDF', 'EPUB', 'CBZ', 'CBT', 'CBR', 'DJVU'}
+    extensions_reader = {'TXT', 'PDF', 'EPUB', 'CBZ', 'CBT', 'CBR', 'DJVU', 'DJV'}
     book_formats = list()
     if len(entry.data):
         for ele in iter(entry.data):
@@ -222,7 +227,7 @@ def send_mail(book_id, book_format, convert, ereader_mail, calibrepath, user_id)
             email_text = N_("%(book)s send to eReader", book=link)
             WorkerThread.add(user_id, TaskEmail(_("Send to eReader"), book.path, converted_file_name,
                              config.get_mail_settings(), ereader_mail,
-                             email_text, _('This Email has been sent via Calibre-Web.')))
+                             email_text, _('This Email has been sent via Calibre-Web.'),book.id))
             return
     return _("The requested file could not be read. Maybe wrong permissions?")
 
@@ -689,16 +694,18 @@ def valid_password(check_password):
     if config.config_password_policy:
         verify = ""
         if config.config_password_min_length > 0:
-            verify += "^(?=.{" + str(config.config_password_min_length) + ",}$)"
+            verify += r"^(?=.{" + str(config.config_password_min_length) + ",}$)"
         if config.config_password_number:
-            verify += "(?=.*?\d)"
+            verify += r"(?=.*?\d)"
         if config.config_password_lower:
-            verify += "(?=.*?[a-z])"
+            verify += r"(?=.*?[\p{Ll}])"
         if config.config_password_upper:
-            verify += "(?=.*?[A-Z])"
+            verify += r"(?=.*?[\p{Lu}])"
+        if config.config_password_character:
+            verify += r"(?=.*?[\p{Letter}])"
         if config.config_password_special:
-            verify += "(?=.*?[^A-Za-z\s0-9])"
-        match = re.match(verify, check_password)
+            verify += r"(?=.*?[^\p{Letter}\s0-9])"
+        match = regex.match(verify, check_password)
         if not match:
             raise Exception(_("Password doesn't comply with password validation rules"))
     return check_password
@@ -732,28 +739,27 @@ def delete_book(book, calibrepath, book_format):
         return delete_book_file(book, calibrepath, book_format)
 
 
-def get_cover_on_failure(use_generic_cover):
-    if use_generic_cover:
-        try:
-            return send_from_directory(_STATIC_DIR, "generic_cover.jpg")
-        except PermissionError:
-            log.error("No permission to access generic_cover.jpg file.")
-            abort(403)
-    abort(404)
+def get_cover_on_failure():
+    try:
+        return send_from_directory(_STATIC_DIR, "generic_cover.jpg")
+    except PermissionError:
+        log.error("No permission to access generic_cover.jpg file.")
+        abort(403)
 
 
 def get_book_cover(book_id, resolution=None):
     book = calibre_db.get_filtered_book(book_id, allow_show_archived=True)
-    return get_book_cover_internal(book, use_generic_cover_on_failure=True, resolution=resolution)
+    return get_book_cover_internal(book, resolution=resolution)
 
 
-# Called only by kobo sync -> cover not found should be answered with 404 and not with default cover
 def get_book_cover_with_uuid(book_uuid, resolution=None):
     book = calibre_db.get_book_by_uuid(book_uuid)
-    return get_book_cover_internal(book, use_generic_cover_on_failure=False, resolution=resolution)
+    if not book:
+        return  # allows kobo.HandleCoverImageRequest to proxy request
+    return get_book_cover_internal(book, resolution=resolution)
 
 
-def get_book_cover_internal(book, use_generic_cover_on_failure, resolution=None):
+def get_book_cover_internal(book, resolution=None):
     if book and book.has_cover:
 
         # Send the book cover thumbnail if it exists in cache
@@ -769,26 +775,26 @@ def get_book_cover_internal(book, use_generic_cover_on_failure, resolution=None)
         if config.config_use_google_drive:
             try:
                 if not gd.is_gdrive_ready():
-                    return get_cover_on_failure(use_generic_cover_on_failure)
+                    return get_cover_on_failure()
                 path = gd.get_cover_via_gdrive(book.path)
                 if path:
                     return redirect(path)
                 else:
                     log.error('{}/cover.jpg not found on Google Drive'.format(book.path))
-                    return get_cover_on_failure(use_generic_cover_on_failure)
+                    return get_cover_on_failure()
             except Exception as ex:
                 log.error_or_exception(ex)
-                return get_cover_on_failure(use_generic_cover_on_failure)
+                return get_cover_on_failure()
 
         # Send the book cover from the Calibre directory
         else:
-            cover_file_path = os.path.join(config.config_calibre_dir, book.path)
+            cover_file_path = os.path.join(config.get_book_path(), book.path)
             if os.path.isfile(os.path.join(cover_file_path, "cover.jpg")):
                 return send_from_directory(cover_file_path, "cover.jpg")
             else:
-                return get_cover_on_failure(use_generic_cover_on_failure)
+                return get_cover_on_failure()
     else:
-        return get_cover_on_failure(use_generic_cover_on_failure)
+        return get_cover_on_failure()
 
 
 def get_book_cover_thumbnail(book, resolution):
@@ -811,7 +817,7 @@ def get_series_thumbnail_on_failure(series_id, resolution):
         .filter(db.Books.has_cover == 1) \
         .first()
 
-    return get_book_cover_internal(book, use_generic_cover_on_failure=True, resolution=resolution)
+    return get_book_cover_internal(book, resolution=resolution)
 
 
 def get_series_cover_thumbnail(series_id, resolution=None):
@@ -922,10 +928,7 @@ def save_cover(img, book_path):
             return False, _("Only jpg/jpeg files are supported as coverfile")
 
     if config.config_use_google_drive:
-        tmp_dir = os.path.join(gettempdir(), 'calibre_web')
-
-        if not os.path.isdir(tmp_dir):
-            os.mkdir(tmp_dir)
+        tmp_dir = get_temp_dir()
         ret, message = save_cover_from_filestorage(tmp_dir, "uploaded_cover.jpg", img)
         if ret is True:
             gd.uploadFileToEbooksFolder(os.path.join(book_path, 'cover.jpg').replace("\\", "/"),
@@ -935,33 +938,72 @@ def save_cover(img, book_path):
         else:
             return False, message
     else:
-        return save_cover_from_filestorage(os.path.join(config.config_calibre_dir, book_path), "cover.jpg", img)
+        return save_cover_from_filestorage(os.path.join(config.get_book_path(), book_path), "cover.jpg", img)
 
 
 def do_download_file(book, book_format, client, data, headers):
+    book_name = data.name
     if config.config_use_google_drive:
         # startTime = time.time()
-        df = gd.getFileFromEbooksFolder(book.path, data.name + "." + book_format)
+        df = gd.getFileFromEbooksFolder(book.path, book_name + "." + book_format)
         # log.debug('%s', time.time() - startTime)
         if df:
-            return gd.do_gdrive_download(df, headers)
+            if config.config_embed_metadata and (
+                 (book_format == "kepub" and config.config_kepubifypath ) or
+                 (book_format != "kepub" and config.config_binariesdir)):
+                output_path = os.path.join(config.config_calibre_dir, book.path)
+                if not os.path.exists(output_path):
+                    os.makedirs(output_path)
+                output = os.path.join(config.config_calibre_dir, book.path, book_name + "." + book_format)
+                gd.downloadFile(book.path, book_name + "." + book_format, output)
+                if book_format == "kepub" and config.config_kepubifypath:
+                    filename, download_name = do_kepubify_metadata_replace(book, output)
+                elif book_format != "kepub" and config.config_binariesdir:
+                    filename, download_name = do_calibre_export(book.id, book_format)
+            else:
+                return gd.do_gdrive_download(df, headers)
         else:
             abort(404)
     else:
-        filename = os.path.join(config.config_calibre_dir, book.path)
-        if not os.path.isfile(os.path.join(filename, data.name + "." + book_format)):
+        filename = os.path.join(config.get_book_path(), book.path)
+        if not os.path.isfile(os.path.join(filename, book_name + "." + book_format)):
             # ToDo: improve error handling
-            log.error('File not found: %s', os.path.join(filename, data.name + "." + book_format))
+            log.error('File not found: %s', os.path.join(filename, book_name + "." + book_format))
 
         if client == "kobo" and book_format == "kepub":
             headers["Content-Disposition"] = headers["Content-Disposition"].replace(".kepub", ".kepub.epub")
 
-        response = make_response(send_from_directory(filename, data.name + "." + book_format))
-        # ToDo Check headers parameter
-        for element in headers:
-            response.headers[element[0]] = element[1]
-        log.info('Downloading file: {}'.format(os.path.join(filename, data.name + "." + book_format)))
-        return response
+        if book_format == "kepub" and config.config_kepubifypath and config.config_embed_metadata:
+            filename, download_name = do_kepubify_metadata_replace(book, os.path.join(filename,
+                                                                                      book_name + "." + book_format))
+        elif book_format != "kepub" and config.config_binariesdir and config.config_embed_metadata:
+                filename, download_name = do_calibre_export(book.id, book_format)
+        else:
+            download_name = book_name
+
+    response = make_response(send_from_directory(filename, download_name + "." + book_format))
+    # ToDo Check headers parameter
+    for element in headers:
+        response.headers[element[0]] = element[1]
+    log.info('Downloading file: {}'.format(os.path.join(filename, book_name + "." + book_format)))
+    return response
+
+
+def do_kepubify_metadata_replace(book, file_path):
+    custom_columns = (calibre_db.session.query(db.CustomColumns)
+                      .filter(db.CustomColumns.mark_for_delete == 0)
+                      .filter(db.CustomColumns.datatype.notin_(db.cc_exceptions))
+                      .order_by(db.CustomColumns.label).all())
+
+    tree, cf_name = get_content_opf(file_path)
+    package = create_new_metadata_backup(book, custom_columns, current_user.locale, _("Cover"), lang_type=2)
+    content = replace_metadata(tree, package)
+    tmp_dir = get_temp_dir()
+    temp_file_name = str(uuid4())
+    # open zipfile and replace metadata block in content.opf
+    updateEpub(file_path, os.path.join(tmp_dir, temp_file_name + ".kepub"), cf_name, content)
+    return tmp_dir, temp_file_name
+
 
 ##################################
 
@@ -985,6 +1027,47 @@ def check_unrar(unrar_location):
         return _('Error executing UnRar')
 
 
+def check_calibre(calibre_location):
+    if not calibre_location:
+        return
+
+    if not os.path.exists(calibre_location):
+        return _('Could not find the specified directory')
+
+    if not os.path.isdir(calibre_location):
+        return _('Please specify a directory, not a file')
+
+    try:
+        supported_binary_paths = [os.path.join(calibre_location, binary)
+                                  for binary in SUPPORTED_CALIBRE_BINARIES.values()]
+        binaries_available = [os.path.isfile(binary_path) for binary_path in supported_binary_paths]
+        binaries_executable = [os.access(binary_path, os.X_OK) for binary_path in supported_binary_paths]
+        if all(binaries_available) and all(binaries_executable):
+            values = [process_wait([binary_path, "--version"], pattern=r'\(calibre (.*)\)')
+                      for binary_path in supported_binary_paths]
+            if all(values):
+                version = values[0].group(1)
+                log.debug("calibre version %s", version)
+            else:
+                return _('Calibre binaries not viable')
+        else:
+            ret_val = []
+            missing_binaries=[path for path, available in
+                              zip(SUPPORTED_CALIBRE_BINARIES.values(), binaries_available) if not available]
+
+            missing_perms=[path for path, available in
+                           zip(SUPPORTED_CALIBRE_BINARIES.values(), binaries_executable) if not available]
+            if missing_binaries:
+                ret_val.append(_('Missing calibre binaries: %(missing)s', missing=", ".join(missing_binaries)))
+            if missing_perms:
+                ret_val.append(_('Missing executable permissions: %(missing)s', missing=", ".join(missing_perms)))
+            return ", ".join(ret_val)
+
+    except (OSError, UnicodeDecodeError) as err:
+        log.error_or_exception(err)
+        return _('Error excecuting Calibre')
+
+
 def json_serial(obj):
     """JSON serializer for objects not serializable by default json code"""
 
@@ -1009,43 +1092,38 @@ def tags_filters():
 
 
 # checks if domain is in database (including wildcards)
-# example SELECT * FROM @TABLE WHERE  'abcdefg' LIKE Name;
+# example SELECT * FROM @TABLE WHERE 'abcdefg' LIKE Name;
 # from https://code.luasoftware.com/tutorials/flask/execute-raw-sql-in-flask-sqlalchemy/
 # in all calls the email address is checked for validity
 def check_valid_domain(domain_text):
     sql = "SELECT * FROM registration WHERE (:domain LIKE domain and allow = 1);"
-    result = ub.session.query(ub.Registration).from_statement(text(sql)).params(domain=domain_text).all()
-    if not len(result):
+    if not len(ub.session.query(ub.Registration).from_statement(text(sql)).params(domain=domain_text).all()):
         return False
     sql = "SELECT * FROM registration WHERE (:domain LIKE domain and allow = 0);"
-    result = ub.session.query(ub.Registration).from_statement(text(sql)).params(domain=domain_text).all()
-    return not len(result)
+    return not len(ub.session.query(ub.Registration).from_statement(text(sql)).params(domain=domain_text).all())
 
 
 def get_download_link(book_id, book_format, client):
     book_format = book_format.split(".")[0]
     book = calibre_db.get_filtered_book(book_id, allow_show_archived=True)
-    data1= ""
     if book:
         data1 = calibre_db.get_book_format(book.id, book_format.upper())
+        if data1:
+            # collect downloaded books only for registered user and not for anonymous user
+            if current_user.is_authenticated:
+                ub.update_download(book_id, int(current_user.id))
+            file_name = book.title
+            if len(book.authors) > 0:
+                file_name = file_name + ' - ' + book.authors[0].name
+            file_name = get_valid_filename(file_name, replace_whitespace=False)
+            headers = Headers()
+            headers["Content-Type"] = mimetypes.types_map.get('.' + book_format, "application/octet-stream")
+            headers["Content-Disposition"] = "attachment; filename=%s.%s; filename*=UTF-8''%s.%s" % (
+                quote(file_name), book_format, quote(file_name), book_format)
+            return do_download_file(book, book_format, client, data1, headers)
     else:
         log.error("Book id {} not found for downloading".format(book_id))
-        abort(404)
-    if data1:
-        # collect downloaded books only for registered user and not for anonymous user
-        if current_user.is_authenticated:
-            ub.update_download(book_id, int(current_user.id))
-        file_name = book.title
-        if len(book.authors) > 0:
-            file_name = file_name + ' - ' + book.authors[0].name
-        file_name = get_valid_filename(file_name, replace_whitespace=False)
-        headers = Headers()
-        headers["Content-Type"] = mimetypes.types_map.get('.' + book_format, "application/octet-stream")
-        headers["Content-Disposition"] = "attachment; filename=%s.%s; filename*=UTF-8''%s.%s" % (
-            quote(file_name.encode('utf-8')), book_format, quote(file_name.encode('utf-8')), book_format)
-        return do_download_file(book, book_format, client, data1, headers)
-    else:
-        abort(404)
+    abort(404)
 
 
 def clear_cover_thumbnail_cache(book_id):

cps/iso_language_names.py

@@ -6579,6 +6579,384 @@ LANGUAGE_NAMES = {
         "zxx": "brak kontekstu językowego",
         "zza": "zazaki"
     },
+    "pt": {
+        "abk": "Abcázio",
+        "ace": "Achém",
+        "ach": "Acoli",
+        "ada": "Adangme",
+        "ady": "Adyghe",
+        "aar": "Afar",
+        "afh": "Afrihili",
+        "afr": "Africânder",
+        "ain": "Ainu (Japão)",
+        "aka": "Akan",
+        "akk": "Acadiano",
+        "sqi": "Albanês",
+        "ale": "Aleúte",
+        "amh": "Amárico",
+        "anp": "Angika",
+        "ara": "Arabic",
+        "arg": "Aragonese",
+        "arp": "Arapaho",
+        "arw": "Arawak",
+        "hye": "Armênio",
+        "asm": "Assamese",
+        "ast": "Asturian",
+        "ava": "Avaric",
+        "ave": "Avestan",
+        "awa": "Awadhi",
+        "aym": "Aymara",
+        "aze": "Azerbaijano",
+        "ban": "Balinês",
+        "bal": "Balúchi",
+        "bam": "Bambara",
+        "bas": "Basa (Cameroon)",
+        "bak": "Bashkir",
+        "eus": "Basque",
+        "bej": "Beja",
+        "bel": "Belarusian",
+        "bem": "Bemba (Zambia)",
+        "ben": "Bengali",
+        "bit": "Berinomo",
+        "bho": "Bhojpuri",
+        "bik": "Bikol",
+        "byn": "Bilin",
+        "bin": "Bini",
+        "bis": "Bislama",
+        "zbl": "Blissymbols",
+        "bos": "Bosnian",
+        "bra": "Braj",
+        "bre": "Bretão",
+        "bug": "Buginese",
+        "bul": "Búlgaro",
+        "bua": "Buriat",
+        "mya": "Birmanês",
+        "cad": "Caddo",
+        "cat": "Catalão",
+        "ceb": "Cebuano",
+        "chg": "Chagatai",
+        "cha": "Chamorro",
+        "che": "Chechen",
+        "chr": "Cheroqui",
+        "chy": "Cheyenne",
+        "chb": "Chibcha",
+        "zho": "Chinês",
+        "chn": "Chinook jargon",
+        "chp": "Chipewyan",
+        "cho": "Choctaw",
+        "cht": "Cholón",
+        "chk": "Chuukese",
+        "chv": "Chuvash",
+        "cop": "Coptic",
+        "cor": "Cornish",
+        "cos": "Corsican",
+        "cre": "Cree",
+        "mus": "Creek",
+        "hrv": "Croata",
+        "ces": "Czech",
+        "dak": "Dacota",
+        "dan": "Danish",
+        "dar": "Dargwa",
+        "del": "Delaware",
+        "div": "Dhivehi",
+        "din": "Dinka",
+        "doi": "Dogri (macrolanguage)",
+        "dgr": "Dogrib",
+        "dua": "Duala",
+        "nld": "Holandês",
+        "dse": "Língua gestual holandesa",
+        "dyu": "Dyula",
+        "dzo": "Dzongkha",
+        "efi": "Efik",
+        "egy": "Egyptian (Ancient)",
+        "eka": "Ekajuk",
+        "elx": "Elamite",
+        "eng": "Inglês",
+        "enu": "Enu",
+        "myv": "Erzya",
+        "epo": "Esperanto",
+        "est": "Estónio",
+        "ewe": "Ewe",
+        "ewo": "Ewondo",
+        "fan": "Fang (Equatorial Guinea)",
+        "fat": "Fanti",
+        "fao": "Faroese",
+        "fij": "Fijian",
+        "fil": "Filipino",
+        "fin": "Finlandês",
+        "fon": "Fon",
+        "fra": "Francês",
+        "fur": "Friuliano",
+        "ful": "Fulah",
+        "gaa": "Ga",
+        "glg": "Galician",
+        "lug": "Ganda",
+        "gay": "Gayo",
+        "gba": "Gbaya (Central African Republic)",
+        "hmj": "Ge",
+        "gez": "Geez",
+        "kat": "Georgiano",
+        "deu": "Alemão",
+        "gil": "Gilbertês",
+        "gon": "Gondi",
+        "gor": "Gorontalo",
+        "got": "Gótico",
+        "grb": "Grebo",
+        "grn": "Guarani",
+        "guj": "Guzerate",
+        "gwi": "Gwichʼin",
+        "hai": "Haida",
+        "hau": "Hauçá",
+        "haw": "Havaiano",
+        "heb": "Hebraico",
+        "her": "Herero",
+        "hil": "Hiligaynon",
+        "hin": "Hindi",
+        "hmo": "Hiri Motu",
+        "hit": "Hitita",
+        "hmn": "Hmong",
+        "hun": "Húngaro",
+        "hup": "Hupa",
+        "iba": "Iban",
+        "isl": "Islandês",
+        "ido": "Ido",
+        "ibo": "Igbo",
+        "ilo": "Ilocano",
+        "ind": "Indonésio",
+        "inh": "Ingush",
+        "ina": "Interlingua (International Auxiliary Language Association)",
+        "ile": "Interlingue",
+        "iku": "Inuktitut",
+        "ipk": "Inupiaq",
+        "gle": "Irlandês",
+        "ita": "Italiano",
+        "jpn": "Japanese",
+        "jav": "Javanês",
+        "jrb": "Judeo-Arabic",
+        "jpr": "Judeo-Persian",
+        "kbd": "Kabardian",
+        "kab": "Kabyle",
+        "kac": "Kachin",
+        "kal": "Kalaallisut",
+        "xal": "Kalmyk",
+        "kam": "Kamba (Quênia)",
+        "kan": "Canarês",
+        "kau": "Kanuri",
+        "kaa": "Kara-Kalpak",
+        "krc": "Karachay-Balkar",
+        "krl": "Karelian",
+        "kas": "Kashmiri",
+        "csb": "Kashubian",
+        "kaw": "Kawi",
+        "kaz": "Cazaque",
+        "kha": "Khasi",
+        "kho": "Khotanese",
+        "kik": "Quicuio",
+        "kmb": "Quimbundo",
+        "kin": "Kinyarwanda",
+        "kir": "Quirguiz",
+        "tlh": "Klingon",
+        "kom": "Komi",
+        "kon": "Quicongo",
+        "kok": "Konkani (macrolanguage)",
+        "kor": "Coreano",
+        "kos": "Kosraean",
+        "kpe": "Kpelle",
+        "kua": "Kuanyama",
+        "kum": "Kumyk",
+        "kur": "Kurdish",
+        "kru": "Kurukh",
+        "kut": "Kutenai",
+        "lad": "Ladino",
+        "lah": "Lahnda",
+        "lam": "Lamba",
+        "lao": "Laosiano",
+        "lat": "Latin",
+        "lav": "Letão",
+        "lez": "Lezghian",
+        "lim": "Limburgan",
+        "lin": "Lingala",
+        "lit": "Lituano",
+        "jbo": "Lojban",
+        "loz": "Lozi",
+        "lub": "Luba-Catanga",
+        "lua": "Luba-Lulua",
+        "lui": "Luiseno",
+        "smj": "Lule Sami",
+        "lun": "Lunda",
+        "luo": "Luo (Kenya and Tanzania)",
+        "lus": "Lushai",
+        "ltz": "Luxembourgish",
+        "mkd": "Macedónio",
+        "mad": "Madurese",
+        "mag": "Magahi",
+        "mai": "Maithili",
+        "mak": "Makasar",
+        "mlg": "Malgaxe",
+        "msa": "Malay (macrolanguage)",
+        "mal": "Malayalam",
+        "mlt": "Maltese",
+        "mnc": "Manchu",
+        "mdr": "Mandar",
+        "man": "Mandinga",
+        "mni": "Manipuri",
+        "glv": "Manx",
+        "mri": "Maori",
+        "arn": "Mapudungun",
+        "mar": "Marata",
+        "chm": "Mari (Russia)",
+        "mah": "Marshallese",
+        "mwr": "Marwari",
+        "mas": "Masai",
+        "men": "Mende (Sierra Leone)",
+        "mic": "Mi'kmaq",
+        "min": "Minangkabau",
+        "mwl": "Mirandês",
+        "moh": "Mohawk",
+        "mdf": "Mocsa",
+        "lol": "Mongo",
+        "mon": "Mongolian",
+        "mos": "Mossi",
+        "mul": "Múltiplos idiomas",
+        "nqo": "N'Ko",
+        "nau": "Nauruano",
+        "nav": "Navajo",
+        "ndo": "Ndonga",
+        "nap": "Neapolitan",
+        "nia": "Nias",
+        "niu": "Niueano",
+        "zxx": "Sem conteúdo linguistico",
+        "nog": "Nogai",
+        "nor": "Norueguês",
+        "nob": "Norueguês, Dano",
+        "nno": "Norueguês, Novo",
+        "nym": "Nyamwezi",
+        "nya": "Nyanja",
+        "nyn": "Nyankole",
+        "nyo": "Nyoro",
+        "nzi": "Nzima",
+        "oci": "Occitan (post 1500)",
+        "oji": "Ojibwa",
+        "orm": "Oromo",
+        "osa": "Osage",
+        "oss": "Ossetian",
+        "pal": "Pálavi",
+        "pau": "Palauano",
+        "pli": "Pali",
+        "pam": "Pampanga",
+        "pag": "Pangasinense",
+        "pan": "Panjabi",
+        "pap": "Papiamento",
+        "fas": "Persian",
+        "phn": "Fenício",
+        "pon": "Pohnpeian",
+        "pol": "Polaco",
+        "por": "Português",
+        "pus": "Pushto",
+        "que": "Quíchua",
+        "raj": "Rajastani",
+        "rap": "Rapanui",
+        "ron": "Romeno",
+        "roh": "Romansh",
+        "rom": "Romany",
+        "run": "Rundi",
+        "rus": "Russo",
+        "smo": "Samoan",
+        "sad": "Sandawe",
+        "sag": "Sango",
+        "san": "Sanskrit",
+        "sat": "Santali",
+        "srd": "Sardinian",
+        "sas": "Sasak",
+        "sco": "Scots",
+        "sel": "Selkup",
+        "srp": "Sérvio",
+        "srr": "Serere",
+        "shn": "Shan",
+        "sna": "Shona",
+        "scn": "Sicilian",
+        "sid": "Sidamo",
+        "bla": "Siksika",
+        "snd": "Sindi",
+        "sin": "Cingalês",
+        "den": "Slave (Athapascan)",
+        "slk": "Eslovaco",
+        "slv": "Esloveno",
+        "sog": "Sogdian",
+        "som": "Somali",
+        "snk": "Soninke",
+        "spa": "Espanhol",
+        "srn": "Sranan Tongo",
+        "suk": "Sukuma",
+        "sux": "Sumerian",
+        "sun": "Sudanês",
+        "sus": "Sosso",
+        "swa": "Swahili (macrolanguage)",
+        "ssw": "Swati",
+        "swe": "Sueco",
+        "syr": "Siríaco",
+        "tgl": "Tagaloge",
+        "tah": "Tahitian",
+        "tgk": "Tajik",
+        "tmh": "Tamaxeque",
+        "tam": "Tamil",
+        "tat": "Tatar",
+        "tel": "Telugu",
+        "ter": "Tereno",
+        "tet": "Tétum",
+        "tha": "Tailandês",
+        "bod": "Tibetano",
+        "tig": "Tigre",
+        "tir": "Tigrinya",
+        "tem": "Timne",
+        "tiv": "Tiv",
+        "tli": "Tlingit",
+        "tpi": "Tok Pisin",
+        "tkl": "Toquelauano",
+        "tog": "Toganês (Nyasa)",
+        "ton": "Tonga (ilhas tonga)",
+        "tsi": "Tsimshian",
+        "tso": "Tsonga",
+        "tsn": "Tswana",
+        "tum": "Tumbuka",
+        "tur": "Turco",
+        "tuk": "Turcomano",
+        "tvl": "Tuvaluano",
+        "tyv": "Tuvinian",
+        "twi": "Twi",
+        "udm": "Udmurt",
+        "uga": "Ugarítico",
+        "uig": "Uighur",
+        "ukr": "Ucraniano",
+        "umb": "Umbundu",
+        "mis": "Idiomas sem código",
+        "und": "Não identificável",
+        "urd": "Urdu",
+        "uzb": "Usbeque",
+        "vai": "Vai",
+        "ven": "Venda",
+        "vie": "Vietnamita",
+        "vol": "Volapük",
+        "vot": "Votic",
+        "wln": "Walloon",
+        "war": "Waray (Philippines)",
+        "was": "Washo",
+        "cym": "Galês",
+        "wal": "Wolaytta",
+        "wol": "Uolofe",
+        "xho": "Xosa",
+        "sah": "Iacuto",
+        "yao": "Iao",
+        "yap": "Yapese",
+        "yid": "Ídiche",
+        "yor": "Iorubá",
+        "zap": "Zapoteca",
+        "zza": "Zaza",
+        "zen": "Zenaga",
+        "zha": "Zhuang",
+        "zul": "Zulu",
+        "zun": "Zuni"
+    },
     "pt_BR": {
         "abk": "Abcázio",
         "ace": "Achém",
@@ -7382,6 +7760,384 @@ LANGUAGE_NAMES = {
         "zxx": "Нет языкового содержимого",
         "zza": "Зазаки"
     },
+    "sk": {
+        "abk": "Abkhazian",
+        "ace": "Achinese",
+        "ach": "Acoli",
+        "ada": "Adangme",
+        "ady": "Adyghe",
+        "aar": "Afar",
+        "afh": "Afrihili",
+        "afr": "Afrikánsky",
+        "ain": "Ainu (Japan)",
+        "aka": "Akan",
+        "akk": "Akkadian",
+        "sqi": "Albanian",
+        "ale": "Aleut",
+        "amh": "Amharic",
+        "anp": "Angika",
+        "ara": "Arabská",
+        "arg": "Aragonese",
+        "arp": "Arapaho",
+        "arw": "Arawak",
+        "hye": "Arménčina",
+        "asm": "Assamese",
+        "ast": "Asturian",
+        "ava": "Avaric",
+        "ave": "Avestan",
+        "awa": "Awadhi",
+        "aym": "Aymara",
+        "aze": "Ázerbajdžánsky",
+        "ban": "Balinese",
+        "bal": "Baluchi",
+        "bam": "Bambara",
+        "bas": "Basa (Cameroon)",
+        "bak": "Bashkir",
+        "eus": "Baskitský",
+        "bej": "Beja",
+        "bel": "Belarusian",
+        "bem": "Bemba (Zambia)",
+        "ben": "Bengali",
+        "bit": "Berinomo",
+        "bho": "Bhojpuri",
+        "bik": "Bikol",
+        "byn": "Bilin",
+        "bin": "Bini",
+        "bis": "Bislama",
+        "zbl": "Blissymbols",
+        "bos": "Bosnian",
+        "bra": "Braj",
+        "bre": "Bretónsky",
+        "bug": "Buginese",
+        "bul": "Bulharský",
+        "bua": "Buriat",
+        "mya": "Burmese",
+        "cad": "Caddo",
+        "cat": "Katalánsky",
+        "ceb": "Cebuano",
+        "chg": "Chagatai",
+        "cha": "Chamorro",
+        "che": "Chechen",
+        "chr": "Cherokee",
+        "chy": "Cheyenne",
+        "chb": "Chibcha",
+        "zho": "Čínsky",
+        "chn": "Chinook jargon",
+        "chp": "Chipewyan",
+        "cho": "Choctaw",
+        "cht": "Cholón",
+        "chk": "Chuukese",
+        "chv": "Chuvash",
+        "cop": "Coptic",
+        "cor": "Cornish",
+        "cos": "Corsican",
+        "cre": "Cree",
+        "mus": "Creek",
+        "hrv": "Chorvátsky",
+        "ces": "Český",
+        "dak": "Dakota",
+        "dan": "Dánsky",
+        "dar": "Dargwa",
+        "del": "Delaware",
+        "div": "Dhivehi",
+        "din": "Dinka",
+        "doi": "Dogri (macrolanguage)",
+        "dgr": "Dogrib",
+        "dua": "Duala",
+        "nld": "Holandský",
+        "dse": "Dutch Sign Language",
+        "dyu": "Dyula",
+        "dzo": "Dzongkha",
+        "efi": "Efik",
+        "egy": "Egyptian (Ancient)",
+        "eka": "Ekajuk",
+        "elx": "Elamite",
+        "eng": "Angličtina",
+        "enu": "Enu",
+        "myv": "Erzya",
+        "epo": "Esperanto",
+        "est": "Estónsky",
+        "ewe": "Ewe",
+        "ewo": "Ewondo",
+        "fan": "Fang (Equatorial Guinea)",
+        "fat": "Fanti",
+        "fao": "Faroese",
+        "fij": "Fijian",
+        "fil": "Filipino",
+        "fin": "Fínsky",
+        "fon": "Fon",
+        "fra": "Francúzsky",
+        "fur": "Friulian",
+        "ful": "Fulah",
+        "gaa": "Ga",
+        "glg": "Galician",
+        "lug": "Ganda",
+        "gay": "Gayo",
+        "gba": "Gbaya (Central African Republic)",
+        "hmj": "Ge",
+        "gez": "Geez",
+        "kat": "Georgian",
+        "deu": "Nemecký",
+        "gil": "Gilbertese",
+        "gon": "Gondi",
+        "gor": "Gorontalo",
+        "got": "Gothic",
+        "grb": "Grebo",
+        "grn": "Guarani",
+        "guj": "Gujarati",
+        "gwi": "Gwichʼin",
+        "hai": "Haida",
+        "hau": "Hausa",
+        "haw": "Hawaiian",
+        "heb": "Hebrejský",
+        "her": "Herero",
+        "hil": "Hiligaynon",
+        "hin": "Hindi",
+        "hmo": "Hiri Motu",
+        "hit": "Hittite",
+        "hmn": "Hmong",
+        "hun": "Maďarský",
+        "hup": "Hupa",
+        "iba": "Iban",
+        "isl": "Islandský",
+        "ido": "Ido",
+        "ibo": "Igbo",
+        "ilo": "Iloko",
+        "ind": "Indonézsky",
+        "inh": "Ingush",
+        "ina": "Interlingua (International Auxiliary Language Association)",
+        "ile": "Interlingue",
+        "iku": "Inuktitut",
+        "ipk": "Inupiaq",
+        "gle": "Írsky",
+        "ita": "Taliansky",
+        "jpn": "Japonský",
+        "jav": "Javanese",
+        "jrb": "Judeo-Arabic",
+        "jpr": "Judeo-Persian",
+        "kbd": "Kabardian",
+        "kab": "Kabyle",
+        "kac": "Kachin",
+        "kal": "Kalaallisut",
+        "xal": "Kalmyk",
+        "kam": "Kamba (Kenya)",
+        "kan": "Kannada",
+        "kau": "Kanuri",
+        "kaa": "Kara-Kalpak",
+        "krc": "Karachay-Balkar",
+        "krl": "Karelian",
+        "kas": "Kashmiri",
+        "csb": "Kashubian",
+        "kaw": "Kawi",
+        "kaz": "Kazakh",
+        "kha": "Khasi",
+        "kho": "Khotanese",
+        "kik": "Kikuyu",
+        "kmb": "Kimbundu",
+        "kin": "Kinyarwanda",
+        "kir": "Kirghiz",
+        "tlh": "Klingon",
+        "kom": "Komi",
+        "kon": "Kongo",
+        "kok": "Konkani (macrolanguage)",
+        "kor": "Kórejský",
+        "kos": "Kosraean",
+        "kpe": "Kpelle",
+        "kua": "Kuanyama",
+        "kum": "Kumyk",
+        "kur": "Kurdský",
+        "kru": "Kurukh",
+        "kut": "Kutenai",
+        "lad": "Ladino",
+        "lah": "Lahnda",
+        "lam": "Lamba",
+        "lao": "Lao",
+        "lat": "Latin",
+        "lav": "Latvian",
+        "lez": "Lezghian",
+        "lim": "Limburgan",
+        "lin": "Lingala",
+        "lit": "Lotyšský",
+        "jbo": "Lojban",
+        "loz": "Lozi",
+        "lub": "Luba-Katanga",
+        "lua": "Luba-Lulua",
+        "lui": "Luiseno",
+        "smj": "Lule Sami",
+        "lun": "Lunda",
+        "luo": "Luo (Kenya and Tanzania)",
+        "lus": "Lushai",
+        "ltz": "Luxembourgish",
+        "mkd": "Macedónsky",
+        "mad": "Madurese",
+        "mag": "Magahi",
+        "mai": "Maithili",
+        "mak": "Makasar",
+        "mlg": "Malagasy",
+        "msa": "Malay (macrolanguage)",
+        "mal": "Malayalam",
+        "mlt": "Maltézsky",
+        "mnc": "Manchu",
+        "mdr": "Mandar",
+        "man": "Mandingo",
+        "mni": "Manipuri",
+        "glv": "Manx",
+        "mri": "Maori",
+        "arn": "Mapudungun",
+        "mar": "Marathi",
+        "chm": "Mari (Russia)",
+        "mah": "Marshallese",
+        "mwr": "Marwari",
+        "mas": "Masai",
+        "men": "Mende (Sierra Leone)",
+        "mic": "Mi'kmaq",
+        "min": "Minangkabau",
+        "mwl": "Mirandese",
+        "moh": "Mohawk",
+        "mdf": "Moksha",
+        "lol": "Mongo",
+        "mon": "Mongolian",
+        "mos": "Mossi",
+        "mul": "Multiple languages",
+        "nqo": "N'Ko",
+        "nau": "Nauru",
+        "nav": "Navajo",
+        "ndo": "Ndonga",
+        "nap": "Neapolitan",
+        "nia": "Nias",
+        "niu": "Niuean",
+        "zxx": "No linguistic content",
+        "nog": "Nogai",
+        "nor": "Norwegian",
+        "nob": "Norwegian Bokmål",
+        "nno": "Norwegian Nynorsk",
+        "nym": "Nyamwezi",
+        "nya": "Nyanja",
+        "nyn": "Nyankole",
+        "nyo": "Nyoro",
+        "nzi": "Nzima",
+        "oci": "Occitan (post 1500)",
+        "oji": "Ojibwa",
+        "orm": "Oromo",
+        "osa": "Osage",
+        "oss": "Ossetian",
+        "pal": "Pahlavi",
+        "pau": "Palauan",
+        "pli": "Pali",
+        "pam": "Pampanga",
+        "pag": "Pangasinan",
+        "pan": "Panjabi",
+        "pap": "Papiamento",
+        "fas": "Persian",
+        "phn": "Phoenician",
+        "pon": "Pohnpeian",
+        "pol": "Poľský",
+        "por": "Portugalský",
+        "pus": "Pashto",
+        "que": "Quechua",
+        "raj": "Rajasthani",
+        "rap": "Rapanui",
+        "ron": "Rumunský",
+        "roh": "Romansh",
+        "rom": "Romany",
+        "run": "Rundi",
+        "rus": "Ruský",
+        "smo": "Samoan",
+        "sad": "Sandawe",
+        "sag": "Sango",
+        "san": "Sanskrit",
+        "sat": "Santali",
+        "srd": "Sardinian",
+        "sas": "Sasak",
+        "sco": "Scots",
+        "sel": "Selkup",
+        "srp": "Srbský",
+        "srr": "Serer",
+        "shn": "Shan",
+        "sna": "Shona",
+        "scn": "Sicilian",
+        "sid": "Sidamo",
+        "bla": "Siksika",
+        "snd": "Sindhi",
+        "sin": "Sinhala",
+        "den": "Slave (Athapascan)",
+        "slk": "Slovenský",
+        "slv": "Slovinský",
+        "sog": "Sogdian",
+        "som": "Somali",
+        "snk": "Soninke",
+        "spa": "Španielsky",
+        "srn": "Sranan Tongo",
+        "suk": "Sukuma",
+        "sux": "Sumerian",
+        "sun": "Sundanese",
+        "sus": "Susu",
+        "swa": "Swahili (macrolanguage)",
+        "ssw": "Swati",
+        "swe": "Švédsky",
+        "syr": "Syriac",
+        "tgl": "Tagalog",
+        "tah": "Tahitian",
+        "tgk": "Tajik",
+        "tmh": "Tamashek",
+        "tam": "Tamilský",
+        "tat": "Tatar",
+        "tel": "Telugu",
+        "ter": "Tereno",
+        "tet": "Tetum",
+        "tha": "Thajský",
+        "bod": "Tibetan",
+        "tig": "Tigre",
+        "tir": "Tigrinya",
+        "tem": "Timne",
+        "tiv": "Tiv",
+        "tli": "Tlingit",
+        "tpi": "Tok Pisin",
+        "tkl": "Tokelau",
+        "tog": "Tonga (Nyasa)",
+        "ton": "Tonga (Tonga Islands)",
+        "tsi": "Tsimshian",
+        "tso": "Tsonga",
+        "tsn": "Tswana",
+        "tum": "Tumbuka",
+        "tur": "Turecký",
+        "tuk": "Turkmen",
+        "tvl": "Tuvalu",
+        "tyv": "Tuvinian",
+        "twi": "Twi",
+        "udm": "Udmurt",
+        "uga": "Ugaritic",
+        "uig": "Uighur",
+        "ukr": "Ukrainian",
+        "umb": "Umbundu",
+        "mis": "Uncoded languages",
+        "und": "Undetermined",
+        "urd": "Urdu",
+        "uzb": "Uzbek",
+        "vai": "Vai",
+        "ven": "Venda",
+        "vie": "Vietnamský",
+        "vol": "Volapük",
+        "vot": "Votic",
+        "wln": "Vallónsky",
+        "war": "Waray (Philippines)",
+        "was": "Washo",
+        "cym": "Welšský",
+        "wal": "Wolaytta",
+        "wol": "Wolof",
+        "xho": "Xhosa",
+        "sah": "Yakut",
+        "yao": "Yao",
+        "yap": "Yapese",
+        "yid": "Yiddish",
+        "yor": "Yoruba",
+        "zap": "Zapotec",
+        "zza": "Zaza",
+        "zen": "Zenaga",
+        "zha": "Zhuang",
+        "zul": "Zulu",
+        "zun": "Zuni"
+    },
     "sv": {
         "aar": "Afar",
         "abk": "Abchaziska",

cps/jinjia.py

@@ -124,7 +124,7 @@ def formatseriesindex_filter(series_index):
                 return int(series_index)
             else:
                 return series_index
-        except ValueError:
+        except (ValueError, TypeError):
             return series_index
     return 0
 

cps/kobo.py

@@ -48,7 +48,7 @@ import requests
 from . import config, logger, kobo_auth, db, calibre_db, helper, shelf as shelf_lib, ub, csrf, kobo_sync_status
 from . import isoLanguages
 from .epub import get_epub_layout
-from .constants import sqlalchemy_version2, COVER_THUMBNAIL_SMALL
+from .constants import COVER_THUMBNAIL_SMALL #, sqlalchemy_version2
 from .helper import get_download_link
 from .services import SyncToken as SyncToken
 from .web import download_required
@@ -56,7 +56,7 @@ from .kobo_auth import requires_kobo_auth, get_auth_token
 
 KOBO_FORMATS = {"KEPUB": ["KEPUB"], "EPUB": ["EPUB3", "EPUB"]}
 KOBO_STOREAPI_URL = "https://storeapi.kobo.com"
-KOBO_IMAGEHOST_URL = "https://kbimages1-a.akamaihd.net"
+KOBO_IMAGEHOST_URL = "https://cdn.kobo.com/book-images"
 
 SYNC_ITEM_LIMIT = 100
 
@@ -137,11 +137,15 @@ def convert_to_kobo_timestamp_string(timestamp):
 
 @kobo.route("/v1/library/sync")
 @requires_kobo_auth
-@download_required
+# @download_required
 def HandleSyncRequest():
+    if not current_user.role_download():
+        log.info("Users need download permissions for syncing library to Kobo reader")
+        return abort(403)
     sync_token = SyncToken.SyncToken.from_headers(request.headers)
-    log.info("Kobo library sync request received.")
+    log.info("Kobo library sync request received")
     log.debug("SyncToken: {}".format(sync_token))
+    log.debug("Download link format {}".format(get_download_url_for_book('[bookid]','[bookformat]')))
     if not current_app.wsgi_app.is_proxied:
         log.debug('Kobo: Received unproxied request, changed request port to external server port')
 
@@ -165,16 +169,10 @@ def HandleSyncRequest():
     only_kobo_shelves = current_user.kobo_only_shelves_sync
 
     if only_kobo_shelves:
-        if sqlalchemy_version2:
-            changed_entries = select(db.Books,
-                                     ub.ArchivedBook.last_modified,
-                                     ub.BookShelf.date_added,
-                                     ub.ArchivedBook.is_archived)
-        else:
-            changed_entries = calibre_db.session.query(db.Books,
-                                                       ub.ArchivedBook.last_modified,
-                                                       ub.BookShelf.date_added,
-                                                       ub.ArchivedBook.is_archived)
+        changed_entries = calibre_db.session.query(db.Books,
+                                                   ub.ArchivedBook.last_modified,
+                                                   ub.BookShelf.date_added,
+                                                   ub.ArchivedBook.is_archived)
         changed_entries = (changed_entries
                            .join(db.Data).outerjoin(ub.ArchivedBook, and_(db.Books.id == ub.ArchivedBook.book_id,
                                                                           ub.ArchivedBook.user_id == current_user.id))
@@ -191,12 +189,9 @@ def HandleSyncRequest():
                            .filter(ub.Shelf.kobo_sync)
                            .distinct())
     else:
-        if sqlalchemy_version2:
-            changed_entries = select(db.Books, ub.ArchivedBook.last_modified, ub.ArchivedBook.is_archived)
-        else:
-            changed_entries = calibre_db.session.query(db.Books,
-                                                       ub.ArchivedBook.last_modified,
-                                                       ub.ArchivedBook.is_archived)
+        changed_entries = calibre_db.session.query(db.Books,
+                                                   ub.ArchivedBook.last_modified,
+                                                   ub.ArchivedBook.is_archived)
         changed_entries = (changed_entries
                            .join(db.Data).outerjoin(ub.ArchivedBook, and_(db.Books.id == ub.ArchivedBook.book_id,
                                                                           ub.ArchivedBook.user_id == current_user.id))
@@ -208,15 +203,12 @@ def HandleSyncRequest():
                            .order_by(db.Books.id))
 
     reading_states_in_new_entitlements = []
-    if sqlalchemy_version2:
-        books = calibre_db.session.execute(changed_entries.limit(SYNC_ITEM_LIMIT))
-    else:
-        books = changed_entries.limit(SYNC_ITEM_LIMIT)
+    books = changed_entries.limit(SYNC_ITEM_LIMIT)
     log.debug("Books to Sync: {}".format(len(books.all())))
     for book in books:
         formats = [data.format for data in book.Books.data]
         if 'KEPUB' not in formats and config.config_kepubifypath and 'EPUB' in formats:
-            helper.convert_book_format(book.Books.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.name)
+            helper.convert_book_format(book.Books.id, config.get_book_path(), 'EPUB', 'KEPUB', current_user.name)
 
         kobo_reading_state = get_or_create_reading_state(book.Books.id)
         entitlement = {
@@ -229,7 +221,7 @@ def HandleSyncRequest():
             new_reading_state_last_modified = max(new_reading_state_last_modified, kobo_reading_state.last_modified)
             reading_states_in_new_entitlements.append(book.Books.id)
 
-        ts_created = book.Books.timestamp
+        ts_created = book.Books.timestamp.replace(tzinfo=None)
 
         try:
             ts_created = max(ts_created, book.date_added)
@@ -242,7 +234,7 @@ def HandleSyncRequest():
             sync_results.append({"ChangedEntitlement": entitlement})
 
         new_books_last_modified = max(
-            book.Books.last_modified, new_books_last_modified
+            book.Books.last_modified.replace(tzinfo=None), new_books_last_modified
         )
         try:
             new_books_last_modified = max(
@@ -254,27 +246,16 @@ def HandleSyncRequest():
         new_books_last_created = max(ts_created, new_books_last_created)
         kobo_sync_status.add_synced_books(book.Books.id)
 
-    if sqlalchemy_version2:
-        max_change = calibre_db.session.execute(changed_entries
-                                                .filter(ub.ArchivedBook.is_archived)
-                                                .filter(ub.ArchivedBook.user_id == current_user.id)
-                                                .order_by(func.datetime(ub.ArchivedBook.last_modified).desc()))\
-            .columns(db.Books).first()
-    else:
-        max_change = changed_entries.from_self().filter(ub.ArchivedBook.is_archived)\
-            .filter(ub.ArchivedBook.user_id == current_user.id) \
-            .order_by(func.datetime(ub.ArchivedBook.last_modified).desc()).first()
+    max_change = changed_entries.filter(ub.ArchivedBook.is_archived)\
+        .filter(ub.ArchivedBook.user_id == current_user.id) \
+        .order_by(func.datetime(ub.ArchivedBook.last_modified).desc()).first()
 
     max_change = max_change.last_modified if max_change else new_archived_last_modified
 
     new_archived_last_modified = max(new_archived_last_modified, max_change)
 
     # no. of books returned
-    if sqlalchemy_version2:
-        entries = calibre_db.session.execute(changed_entries).all()
-        book_count = len(entries)
-    else:
-        book_count = changed_entries.count()
+    book_count = changed_entries.count()
     # last entry:
     cont_sync = bool(book_count)
     log.debug("Remaining books to Sync: {}".format(book_count))
@@ -337,7 +318,7 @@ def generate_sync_response(sync_token, sync_results, set_cont=False):
             extra_headers["x-kobo-recent-reads"] = store_response.headers.get("x-kobo-recent-reads")
 
         except Exception as ex:
-            log.error("Failed to receive or parse response from Kobo's sync endpoint: {}".format(ex))
+            log.error_or_exception("Failed to receive or parse response from Kobo's sync endpoint: {}".format(ex))
     if set_cont:
         extra_headers["x-kobo-sync"] = "continue"
     sync_token.to_headers(extra_headers)
@@ -367,7 +348,7 @@ def HandleMetadataRequest(book_uuid):
     return response
 
 
-def get_download_url_for_book(book, book_format):
+def get_download_url_for_book(book_id, book_format):
     if not current_app.wsgi_app.is_proxied:
         if ':' in request.host and not request.host.endswith(']'):
             host = "".join(request.host.split(':')[:-1])
@@ -379,13 +360,13 @@ def get_download_url_for_book(book, book_format):
             url_base=host,
             url_port=config.config_external_port,
             auth_token=get_auth_token(),
-            book_id=book.id,
+            book_id=book_id,
             book_format=book_format.lower()
         )
     return url_for(
         "kobo.download_book",
         auth_token=kobo_auth.get_auth_token(),
-        book_id=book.id,
+        book_id=book_id,
         book_format=book_format.lower(),
         _external=True,
     )
@@ -468,7 +449,7 @@ def get_metadata(book):
                     {
                         "Format": kobo_format,
                         "Size": book_data.uncompressed_size,
-                        "Url": get_download_url_for_book(book, book_data.format),
+                        "Url": get_download_url_for_book(book.id, book_data.format),
                         # The Kobo forma accepts platforms: (Generic, Android)
                         "Platform": "Generic",
                         # "DrmType": "None", # Not required
@@ -522,7 +503,7 @@ def get_metadata(book):
 @requires_kobo_auth
 # Creates a Shelf with the given items, and returns the shelf's uuid.
 def HandleTagCreate():
-    # catch delete requests, otherwise the are handled in the book delete handler
+    # catch delete requests, otherwise they are handled in the book delete handler
     if request.method == "DELETE":
         abort(405)
     name, items = None, None
@@ -716,20 +697,12 @@ def sync_shelves(sync_token, sync_results, only_kobo_shelves=False):
             })
         extra_filters.append(ub.Shelf.kobo_sync)
 
-    if sqlalchemy_version2:
-        shelflist = ub.session.execute(select(ub.Shelf).outerjoin(ub.BookShelf).filter(
-            or_(func.datetime(ub.Shelf.last_modified) > sync_token.tags_last_modified,
-                func.datetime(ub.BookShelf.date_added) > sync_token.tags_last_modified),
-            ub.Shelf.user_id == current_user.id,
-            *extra_filters
-        ).distinct().order_by(func.datetime(ub.Shelf.last_modified).asc())).columns(ub.Shelf)
-    else:
-        shelflist = ub.session.query(ub.Shelf).outerjoin(ub.BookShelf).filter(
-            or_(func.datetime(ub.Shelf.last_modified) > sync_token.tags_last_modified,
-                func.datetime(ub.BookShelf.date_added) > sync_token.tags_last_modified),
-            ub.Shelf.user_id == current_user.id,
-            *extra_filters
-        ).distinct().order_by(func.datetime(ub.Shelf.last_modified).asc())
+    shelflist = ub.session.query(ub.Shelf).outerjoin(ub.BookShelf).filter(
+        or_(func.datetime(ub.Shelf.last_modified) > sync_token.tags_last_modified,
+            func.datetime(ub.BookShelf.date_added) > sync_token.tags_last_modified),
+        ub.Shelf.user_id == current_user.id,
+        *extra_filters
+    ).distinct().order_by(func.datetime(ub.Shelf.last_modified).asc())
 
     for shelf in shelflist:
         if not shelf_lib.check_shelf_view_permissions(shelf):
@@ -930,20 +903,26 @@ def get_current_bookmark_response(current_bookmark):
 @kobo.route("/<book_uuid>/<width>/<height>/<Quality>/<isGreyscale>/image.jpg")
 @requires_kobo_auth
 def HandleCoverImageRequest(book_uuid, width, height, Quality, isGreyscale):
-    book_cover = helper.get_book_cover_with_uuid(book_uuid, resolution=COVER_THUMBNAIL_SMALL)
-    if not book_cover:
-        if config.config_kobo_proxy:
-            log.debug("Cover for unknown book: %s proxied to kobo" % book_uuid)
-            return redirect(KOBO_IMAGEHOST_URL +
-                            "/{book_uuid}/{width}/{height}/false/image.jpg".format(book_uuid=book_uuid,
-                                                                                   width=width,
-                                                                                   height=height), 307)
-        else:
-            log.debug("Cover for unknown book: %s requested" % book_uuid)
-            # additional proxy request make no sense, -> direct return
-            return make_response(jsonify({}))
-    log.debug("Cover request received for book %s" % book_uuid)
-    return book_cover
+    try:
+        resolution = None if int(height) > 1000 else COVER_THUMBNAIL_SMALL
+    except ValueError:
+        log.error("Requested height %s of book %s is invalid" % (book_uuid, height))
+        resolution = COVER_THUMBNAIL_SMALL
+    book_cover = helper.get_book_cover_with_uuid(book_uuid, resolution=resolution)
+    if book_cover:
+        log.debug("Serving local cover image of book %s" % book_uuid)
+        return book_cover
+
+    if not config.config_kobo_proxy:
+        log.debug("Returning 404 for cover image of unknown book %s" % book_uuid)
+        # additional proxy request make no sense, -> direct return
+        return abort(404)
+
+    log.debug("Redirecting request for cover image of unknown book %s to Kobo" % book_uuid)
+    return redirect(KOBO_IMAGEHOST_URL +
+                    "/{book_uuid}/{width}/{height}/false/image.jpg".format(book_uuid=book_uuid,
+                                                                            width=width,
+                                                                            height=height), 307)
 
 
 @kobo.route("")
@@ -983,6 +962,7 @@ def HandleUnimplementedRequest(dummy=None):
 @kobo.route("/v1/user/wishlist", methods=["GET", "POST"])
 @kobo.route("/v1/user/recommendations", methods=["GET", "POST"])
 @kobo.route("/v1/analytics/<dummy>", methods=["GET", "POST"])
+@kobo.route("/v1/assets", methods=["GET"])
 def HandleUserRequest(dummy=None):
     log.debug("Unimplemented User Request received: %s (request is forwarded to kobo if configured)", request.base_url)
     return redirect_or_proxy_request()
@@ -1041,7 +1021,7 @@ def make_calibre_web_auth_response():
                 "RefreshToken": RefreshToken,
                 "TokenType": "Bearer",
                 "TrackingId": str(uuid.uuid4()),
-                "UserKey": content['UserKey'],
+                "UserKey": content.get('UserKey',""),
             }
         )
     )

cps/kobo_auth.py

@@ -156,6 +156,9 @@ def requires_kobo_auth(f):
                 limiter.check()
             except RateLimitExceeded:
                 return abort(429)
+            except (ConnectionError, Exception) as e:
+                log.error("Connection error to limiter backend: %s", e)
+                return abort(429)
             user = (
                 ub.session.query(ub.User)
                 .join(ub.RemoteAuthToken)

cps/logger.py

@@ -150,7 +150,7 @@ def setup(log_file, log_level=None):
     else:
         try:
             file_handler = RotatingFileHandler(log_file, maxBytes=100000, backupCount=2, encoding='utf-8')
-        except IOError:
+        except (IOError, PermissionError):
             if log_file == DEFAULT_LOG_FILE:
                 raise
             file_handler = RotatingFileHandler(DEFAULT_LOG_FILE, maxBytes=100000, backupCount=2, encoding='utf-8')
@@ -177,7 +177,7 @@ def create_access_log(log_file, log_name, formatter):
     access_log.setLevel(logging.INFO)
     try:
         file_handler = RotatingFileHandler(log_file, maxBytes=50000, backupCount=2, encoding='utf-8')
-    except IOError:
+    except (IOError, PermissionError):
         if log_file == DEFAULT_ACCESS_LOG:
             raise
         file_handler = RotatingFileHandler(DEFAULT_ACCESS_LOG, maxBytes=50000, backupCount=2, encoding='utf-8')

cps/metadata_provider/amazon.py

@@ -98,7 +98,7 @@ class Amazon(Metadata):
                     try:
                         match.authors = [next(
                             filter(lambda i: i != " " and i != "\n" and not i.startswith("{"),
-                                   x.findAll(text=True))).strip()
+                                   x.findAll(string=True))).strip()
                                         for x in soup2.findAll("span", attrs={"class": "author"})]
                     except (AttributeError, TypeError, StopIteration):
                         match.authors = ""

cps/metadata_provider/douban.py

@@ -169,7 +169,8 @@ class Douban(Metadata):
             ),
         )
 
-        html = etree.HTML(r.content.decode("utf8"))
+        decode_content = r.content.decode("utf8")
+        html = etree.HTML(decode_content)
 
         match.title = html.xpath(self.TITTLE_XPATH)[0].text
         match.cover = html.xpath(
@@ -184,7 +185,7 @@ class Douban(Metadata):
         if len(tag_elements):
             match.tags = [tag_element.text for tag_element in tag_elements]
         else:
-            match.tags = self._get_tags(html.text)
+            match.tags = self._get_tags(decode_content)
 
         description_element = html.xpath(self.DESCRIPTION_XPATH)
         if len(description_element):

cps/metadata_provider/lubimyczytac.py

@@ -97,12 +97,14 @@ class LubimyCzytac(Metadata):
     LANGUAGES = f"{CONTAINER}//dt[contains(text(),'Język:')]{SIBLINGS}/text()"
     DESCRIPTION = f"{CONTAINER}//div[@class='collapse-content']"
     SERIES = f"{CONTAINER}//span/a[contains(@href,'/cykl/')]/text()"
+    TRANSLATOR = f"{CONTAINER}//dt[contains(text(),'Tłumacz:')]{SIBLINGS}/a/text()"
 
     DETAILS = "//div[@id='book-details']"
     PUBLISH_DATE = "//dt[contains(@title,'Data pierwszego wydania"
     FIRST_PUBLISH_DATE = f"{DETAILS}{PUBLISH_DATE} oryginalnego')]{SIBLINGS}[1]/text()"
     FIRST_PUBLISH_DATE_PL = f"{DETAILS}{PUBLISH_DATE} polskiego')]{SIBLINGS}[1]/text()"
-    TAGS = "//nav[@aria-label='breadcrumb']//a[contains(@href,'/ksiazki/k/')]/text()"
+    TAGS = "//a[contains(@href,'/ksiazki/t/')]/text()" # "//nav[@aria-label='breadcrumbs']//a[contains(@href,'/ksiazki/k/')]/span/text()"
+
 
     RATING = "//meta[@property='books:rating:value']/@content"
     COVER = "//meta[@property='og:image']/@content"
@@ -158,6 +160,7 @@ class LubimyCzytac(Metadata):
 
 class LubimyCzytacParser:
     PAGES_TEMPLATE = "<p id='strony'>Książka ma {0} stron(y).</p>"
+    TRANSLATOR_TEMPLATE = "<p id='translator'>Tłumacz: {0}</p>"
     PUBLISH_DATE_TEMPLATE = "<p id='pierwsze_wydanie'>Data pierwszego wydania: {0}</p>"
     PUBLISH_DATE_PL_TEMPLATE = (
         "<p id='pierwsze_wydanie'>Data pierwszego wydania w Polsce: {0}</p>"
@@ -346,5 +349,9 @@ class LubimyCzytacParser:
             description += LubimyCzytacParser.PUBLISH_DATE_PL_TEMPLATE.format(
                 first_publish_date_pl.strftime("%d.%m.%Y")
             )
+        translator = self._parse_xpath_node(xpath=LubimyCzytac.TRANSLATOR)
+        if translator:
+            description += LubimyCzytacParser.TRANSLATOR_TEMPLATE.format(translator)
+
 
         return description

cps/opds.py

@@ -21,9 +21,10 @@
 #  along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 import datetime
+import json
 from urllib.parse import unquote_plus
 
-from flask import Blueprint, request, render_template, make_response, abort
+from flask import Blueprint, request, render_template, make_response, abort, Response
 from flask_login import current_user
 from flask_babel import get_locale
 from flask_babel import gettext as _
@@ -55,7 +56,7 @@ def feed_osd():
     return render_xml_template('osd.xml', lang='en-EN')
 
 
-@opds.route("/opds/search", defaults={'query': ""})
+# @opds.route("/opds/search", defaults={'query': ""})
 @opds.route("/opds/search/<path:query>")
 @requires_basic_auth_if_no_ano
 def feed_cc_search(query):
@@ -412,6 +413,17 @@ def get_metadata_calibre_companion(uuid, library):
         return ""
 
 
+@opds.route("/opds/stats")
+@requires_basic_auth_if_no_ano
+def get_database_stats():
+    stat = dict()
+    stat['books'] = calibre_db.session.query(db.Books).count()
+    stat['authors'] = calibre_db.session.query(db.Authors).count()
+    stat['categories'] = calibre_db.session.query(db.Tags).count()
+    stat['series'] = calibre_db.session.query(db.Series).count()
+    return Response(json.dumps(stat), mimetype="application/json")
+
+
 @opds.route("/opds/thumb_240_240/<book_id>")
 @opds.route("/opds/cover_240_240/<book_id>")
 @opds.route("/opds/cover_90_90/<book_id>")
@@ -490,7 +502,7 @@ def render_element_index(database_column, linked_table, folder):
         entries = entries.join(linked_table).join(db.Books)
     entries = entries.filter(calibre_db.common_filters()).group_by(func.upper(func.substr(database_column, 1, 1))).all()
     elements = []
-    if off == 0:
+    if off == 0 and entries:
         elements.append({'id': "00", 'name': _("All")})
         shift = 1
     for entry in entries[

cps/redirect.py

@@ -29,7 +29,7 @@
 
 from urllib.parse import urlparse, urljoin
 
-from flask import request, url_for, redirect
+from flask import request, url_for, redirect, current_app
 
 
 def is_safe_url(target):
@@ -38,16 +38,15 @@ def is_safe_url(target):
     return test_url.scheme in ('http', 'https') and ref_url.netloc == test_url.netloc
 
 
-def get_redirect_target():
-    for target in request.values.get('next'), request.referrer:
-        if not target:
-            continue
-        if is_safe_url(target):
-            return target
+def remove_prefix(text, prefix):
+    if text.startswith(prefix):
+        return text[len(prefix):]
+    return ""
 
 
-def redirect_back(endpoint, **values):
-    target = request.form['next']
-    if not target or not is_safe_url(target):
+def get_redirect_location(next, endpoint, **values):
+    target = next or url_for(endpoint, **values)
+    adapter = current_app.url_map.bind(urlparse(request.host_url).netloc)
+    if not len(adapter.allowed_methods(remove_prefix(target, request.environ.get('HTTP_X_SCRIPT_NAME',"")))):
         target = url_for(endpoint, **values)
-    return redirect(target)
+    return target

cps/schedule.py

@@ -21,6 +21,7 @@ import datetime
 from . import config, constants
 from .services.background_scheduler import BackgroundScheduler, CronTrigger, use_APScheduler
 from .tasks.database import TaskReconnectDatabase
+from .tasks.tempFolder import TaskDeleteTempFolder
 from .tasks.thumbnail import TaskGenerateCoverThumbnails, TaskGenerateSeriesThumbnails, TaskClearCoverThumbnailCache
 from .services.worker import WorkerThread
 from .tasks.metadata_backup import TaskBackupMetadata
@@ -31,6 +32,9 @@ def get_scheduled_tasks(reconnect=True):
     if reconnect:
         tasks.append([lambda: TaskReconnectDatabase(), 'reconnect', False])
 
+    # Delete temp folder
+    tasks.append([lambda: TaskDeleteTempFolder(), 'delete temp', True])
+
     # Generate metadata.opf file for each changed book
     if config.schedule_metadata_backup:
         tasks.append([lambda: TaskBackupMetadata("en"), 'backup metadata', False])
@@ -65,9 +69,12 @@ def register_scheduled_tasks(reconnect=True):
         duration = config.schedule_duration
 
         # Register scheduled tasks
-        scheduler.schedule_tasks(tasks=get_scheduled_tasks(reconnect), trigger=CronTrigger(hour=start))
+        timezone_info = datetime.datetime.now(datetime.timezone.utc).astimezone().tzinfo
+        scheduler.schedule_tasks(tasks=get_scheduled_tasks(reconnect), trigger=CronTrigger(hour=start,
+                                                                                           timezone=timezone_info))
         end_time = calclulate_end_time(start, duration)
-        scheduler.schedule(func=end_scheduled_tasks, trigger=CronTrigger(hour=end_time.hour, minute=end_time.minute),
+        scheduler.schedule(func=end_scheduled_tasks, trigger=CronTrigger(hour=end_time.hour, minute=end_time.minute,
+                                                                         timezone=timezone_info),
                            name="end scheduled task")
 
         # Kick-off tasks, if they should currently be running
@@ -86,6 +93,8 @@ def register_startup_tasks():
         # Ignore tasks that should currently be running, as these will be added when registering scheduled tasks
         if constants.APP_MODE in ['development', 'test'] and not should_task_be_running(start, duration):
             scheduler.schedule_tasks_immediately(tasks=get_scheduled_tasks(False))
+        else:
+            scheduler.schedule_tasks_immediately(tasks=[[lambda: TaskDeleteTempFolder(), 'delete temp', True]])
 
 
 def should_task_be_running(start, duration):

cps/search.py

@@ -35,13 +35,12 @@ search = Blueprint('search', __name__)
 log = logger.create()
 
 
-@search.route("/search", methods=["POST"])
+@search.route("/search", methods=["GET"])
 @login_required_if_no_ano
 def simple_search():
-    term = dict(request.form).get("query")
+    term = request.args.get("query")
     if term:
-        flask_session['query'] = json.dumps(term.strip())
-        return redirect(url_for('web.books_list', data="search", sort_param='stored', query="")) # term.strip()
+        return redirect(url_for('web.books_list', data="search", sort_param='stored', query=term.strip()))
     else:
         return render_title_template('search.html',
                                      searchterm="",
@@ -218,8 +217,8 @@ def extend_search_term(searchterm,
         searchterm.extend([_("Rating <= %(rating)s", rating=rating_high)])
     if rating_low:
         searchterm.extend([_("Rating >= %(rating)s", rating=rating_low)])
-    if read_status:
-        searchterm.extend([_("Read Status = %(status)s", status=read_status)])
+    if read_status != "Any":
+        searchterm.extend([_("Read Status = '%(status)s'", status=read_status)])
     searchterm.extend(ext for ext in tags['include_extension'])
     searchterm.extend(ext for ext in tags['exclude_extension'])
     # handle custom columns
@@ -284,7 +283,7 @@ def render_adv_search_results(term, offset=None, order=None, limit=None):
             cc_present = True
 
     if any(tags.values()) or author_name or book_title or publisher or pub_start or pub_end or rating_low \
-       or rating_high or description or cc_present or read_status:
+       or rating_high or description or cc_present or read_status != "Any":
         search_term, pub_start, pub_end = extend_search_term(search_term,
                                                              author_name,
                                                              book_title,
@@ -303,7 +302,8 @@ def render_adv_search_results(term, offset=None, order=None, limit=None):
             q = q.filter(func.datetime(db.Books.pubdate) > func.datetime(pub_start))
         if pub_end:
             q = q.filter(func.datetime(db.Books.pubdate) < func.datetime(pub_end))
-        q = q.filter(adv_search_read_status(read_status))
+        if read_status != "Any":
+            q = q.filter(adv_search_read_status(read_status))
         if publisher:
             q = q.filter(db.Books.publishers.any(func.lower(db.Publishers.name).ilike("%" + publisher + "%")))
         q = adv_search_tag(q, tags['include_tag'], tags['exclude_tag'])
@@ -376,13 +376,19 @@ def render_prepare_search_form(cc):
 
 
 def render_search_results(term, offset=None, order=None, limit=None):
-    join = db.books_series_link, db.Books.id == db.books_series_link.c.book, db.Series
-    entries, result_count, pagination = calibre_db.get_search_results(term,
-                                                                      config,
-                                                                      offset,
-                                                                      order,
-                                                                      limit,
-                                                                      *join)
+    if term:
+        join = db.books_series_link, db.Books.id == db.books_series_link.c.book, db.Series
+        entries, result_count, pagination = calibre_db.get_search_results(term,
+                                                                          config,
+                                                                          offset,
+                                                                          order,
+                                                                          limit,
+                                                                          *join)
+    else:
+        entries = list()
+        order = [None, None]
+        pagination = result_count = None
+
     return render_title_template('search.html',
                                  searchterm=term,
                                  pagination=pagination,

cps/server.py

@@ -21,12 +21,13 @@ import os
 import errno
 import signal
 import socket
-import subprocess  # nosec
+import asyncio
 
 try:
     from gevent.pywsgi import WSGIServer
     from .gevent_wsgi import MyWSGIHandler
     from gevent.pool import Pool
+    from gevent.socket import socket as GeventSocket
     from gevent import __version__ as _version
     from greenlet import GreenletExit
     import ssl
@@ -36,6 +37,7 @@ except ImportError:
     from .tornado_wsgi import MyWSGIContainer
     from tornado.httpserver import HTTPServer
     from tornado.ioloop import IOLoop
+    from tornado import netutil
     from tornado import version as _version
     VERSION = 'Tornado ' + _version
     _GEVENT = False
@@ -95,7 +97,12 @@ class WebServer(object):
                 log.warning('Cert path: %s', certfile_path)
                 log.warning('Key path:  %s', keyfile_path)
 
-    def _make_gevent_unix_socket(self, socket_file):
+    def _make_gevent_socket_activated(self):
+        # Reuse an already open socket on fd=SD_LISTEN_FDS_START
+        SD_LISTEN_FDS_START = 3
+        return GeventSocket(fileno=SD_LISTEN_FDS_START)
+
+    def _prepare_unix_socket(self, socket_file):
         # the socket file must not exist prior to bind()
         if os.path.exists(socket_file):
             # avoid nuking regular files and symbolic links (could be a mistype or security issue)
@@ -103,35 +110,41 @@ class WebServer(object):
                 raise OSError(errno.EEXIST, os.strerror(errno.EEXIST), socket_file)
             os.remove(socket_file)
 
-        unix_sock = WSGIServer.get_listener(socket_file, family=socket.AF_UNIX)
         self.unix_socket_file = socket_file
 
-        # ensure current user and group have r/w permissions, no permissions for other users
-        # this way the socket can be shared in a semi-secure manner
-        # between the user running calibre-web and the user running the fronting webserver
-        os.chmod(socket_file, 0o660)
-
-        return unix_sock
-
-    def _make_gevent_socket(self):
+    def _make_gevent_listener(self):
         if os.name != 'nt':
+            socket_activated = os.environ.get("LISTEN_FDS")
+            if socket_activated:
+                sock = self._make_gevent_socket_activated()
+                sock_info = sock.getsockname()
+                return sock, "systemd-socket:" + _readable_listen_address(sock_info[0], sock_info[1])
             unix_socket_file = os.environ.get("CALIBRE_UNIX_SOCKET")
             if unix_socket_file:
-                return self._make_gevent_unix_socket(unix_socket_file), "unix:" + unix_socket_file
+                self._prepare_unix_socket(unix_socket_file)
+                unix_sock = WSGIServer.get_listener(unix_socket_file, family=socket.AF_UNIX)
+                # ensure current user and group have r/w permissions, no permissions for other users
+                # this way the socket can be shared in a semi-secure manner
+                # between the user running calibre-web and the user running the fronting webserver
+                os.chmod(unix_socket_file, 0o660)
+
+                return unix_sock, "unix:" + unix_socket_file
 
         if self.listen_address:
-            return (self.listen_address, self.listen_port), None
+            return ((self.listen_address, self.listen_port),
+                    _readable_listen_address(self.listen_address, self.listen_port))
 
         if os.name == 'nt':
             self.listen_address = '0.0.0.0'
-            return (self.listen_address, self.listen_port), None
+            return ((self.listen_address, self.listen_port),
+                    _readable_listen_address(self.listen_address, self.listen_port))
 
         try:
             address = ('::', self.listen_port)
             sock = WSGIServer.get_listener(address, family=socket.AF_INET6)
         except socket.error as ex:
             log.error('%s', ex)
-            log.warning('Unable to listen on "", trying on IPv4 only...')
+            log.warning('Unable to listen on {}, trying on IPv4 only...'.format(address))
             address = ('', self.listen_port)
             sock = WSGIServer.get_listener(address, family=socket.AF_INET)
 
@@ -201,9 +214,7 @@ class WebServer(object):
         ssl_args = self.ssl_args or {}
 
         try:
-            sock, output = self._make_gevent_socket()
-            if output is None:
-                output = _readable_listen_address(self.listen_address, self.listen_port)
+            sock, output = self._make_gevent_listener()
             log.info('Starting Gevent server on %s', output)
             self.wsgiserver = WSGIServer(sock, self.app, log=self.access_logger, handler_class=MyWSGIHandler,
                                          error_log=log,
@@ -228,17 +239,42 @@ class WebServer(object):
         if os.name == 'nt' and sys.version_info > (3, 7):
             import asyncio
             asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
-        log.info('Starting Tornado server on %s', _readable_listen_address(self.listen_address, self.listen_port))
+        try:
+            # Max Buffersize set to 200MB
+            http_server = HTTPServer(MyWSGIContainer(self.app),
+                                     max_buffer_size=209700000,
+                                     ssl_options=self.ssl_args)
 
-        # Max Buffersize set to 200MB
-        http_server = HTTPServer(MyWSGIContainer(self.app),
-                                 max_buffer_size=209700000,
-                                 ssl_options=self.ssl_args)
-        http_server.listen(self.listen_port, self.listen_address)
-        self.wsgiserver = IOLoop.current()
-        self.wsgiserver.start()
-        # wait for stop signal
-        self.wsgiserver.close(True)
+            unix_socket_file = os.environ.get("CALIBRE_UNIX_SOCKET")
+            if os.environ.get("LISTEN_FDS") and os.name != 'nt':
+                SD_LISTEN_FDS_START = 3
+                sock = socket.socket(fileno=SD_LISTEN_FDS_START)
+                http_server.add_socket(sock)
+                sock.setblocking(0)
+                socket_name =sock.getsockname()
+                output = "systemd-socket:" + _readable_listen_address(socket_name[0], socket_name[1])
+            elif unix_socket_file and os.name != 'nt':
+                self._prepare_unix_socket(unix_socket_file)
+                output = "unix:" + unix_socket_file
+                unix_socket = netutil.bind_unix_socket(self.unix_socket_file)
+                http_server.add_socket(unix_socket)
+                # ensure current user and group have r/w permissions, no permissions for other users
+                # this way the socket can be shared in a semi-secure manner
+                # between the user running calibre-web and the user running the fronting webserver
+                os.chmod(self.unix_socket_file, 0o660)
+            else:
+                output = _readable_listen_address(self.listen_address, self.listen_port)
+                http_server.listen(self.listen_port, self.listen_address)
+            log.info('Starting Tornado server on %s', output)
+
+            self.wsgiserver = IOLoop.current()
+            self.wsgiserver.start()
+            # wait for stop signal
+            self.wsgiserver.close(True)
+        finally:
+            if self.unix_socket_file:
+                os.remove(self.unix_socket_file)
+                self.unix_socket_file = None
 
     def start(self):
         try:
@@ -288,4 +324,8 @@ class WebServer(object):
             if _GEVENT:
                 self.wsgiserver.close()
             else:
-                self.wsgiserver.add_callback_from_signal(self.wsgiserver.stop)
+                if restart:
+                    self.wsgiserver.call_later(1.0, self.wsgiserver.stop)
+                else:
+                    self.wsgiserver.asyncio_loop.call_soon_threadsafe(self.wsgiserver.stop)
+

cps/services/SyncToken.py

@@ -19,11 +19,9 @@
 
 import sys
 from base64 import b64decode, b64encode
-from jsonschema import validate, exceptions, __version__
+from jsonschema import validate, exceptions
 from datetime import datetime
 
-from urllib.parse import unquote
-
 from flask import json
 from .. import logger
 
@@ -35,7 +33,7 @@ def b64encode_json(json_data):
     return b64encode(json.dumps(json_data).encode())
 
 
-# Python3 has a timestamp() method we could be calling, however it's not avaiable in python2.
+# Python3 has a timestamp() method we could be calling, however it's not available in python2.
 def to_epoch_timestamp(datetime_object):
     return (datetime_object - datetime(1970, 1, 1)).total_seconds()
 
@@ -49,7 +47,7 @@ def get_datetime_from_json(json_object, field_name):
 
 
 class SyncToken:
-    """ The SyncToken is used to persist state accross requests.
+    """ The SyncToken is used to persist state across requests.
     When serialized over the response headers, the Kobo device will propagate the token onto following
     requests to the service. As an example use-case, the SyncToken is used to detect books that have been added
     to the library since the last time the device synced to the server.

cps/services/goodreads_support.py

@@ -18,16 +18,49 @@
 
 import time
 from functools import reduce
+import requests
+
+from goodreads.client import GoodreadsClient
+from goodreads.request import GoodreadsRequest
+import xmltodict
 
 try:
-    from goodreads.client import GoodreadsClient
+    import Levenshtein
 except ImportError:
-    from betterreads.client import GoodreadsClient
-
-try: import Levenshtein
-except ImportError: Levenshtein = False
+    Levenshtein = False
 
 from .. import logger
+from ..clean_html import clean_string
+
+class my_GoodreadsClient(GoodreadsClient):
+
+    def request(self, *args, **kwargs):
+        """Create a GoodreadsRequest object and make that request"""
+        req = my_GoodreadsRequest(self, *args, **kwargs)
+        return req.request()
+
+class GoodreadsRequestException(Exception):
+    def __init__(self, error_msg, url):
+        self.error_msg = error_msg
+        self.url = url
+
+    def __str__(self):
+        return self.url, ':', self.error_msg
+
+
+class my_GoodreadsRequest(GoodreadsRequest):
+
+    def request(self):
+        resp = requests.get(self.host+self.path, params=self.params,
+                            headers={"User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:125.0) "
+                                                  "Gecko/20100101 Firefox/125.0"})
+        if resp.status_code != 200:
+            raise GoodreadsRequestException(resp.reason, self.path)
+        if self.req_format == 'xml':
+            data_dict = xmltodict.parse(resp.content)
+            return data_dict['GoodreadsResponse']
+        else:
+            raise Exception("Invalid format")
 
 
 log = logger.create()
@@ -38,20 +71,20 @@ _CACHE_TIMEOUT = 23 * 60 * 60  # 23 hours (in seconds)
 _AUTHORS_CACHE = {}
 
 
-def connect(key=None, secret=None, enabled=True):
+def connect(key=None, enabled=True):
     global _client
 
-    if not enabled or not key or not secret:
+    if not enabled or not key:
         _client = None
         return
 
     if _client:
         # make sure the configuration has not changed since last we used the client
-        if _client.client_key != key or _client.client_secret != secret:
+        if _client.client_key != key:
             _client = None
 
     if not _client:
-        _client = GoodreadsClient(key, secret)
+        _client = my_GoodreadsClient(key, None)
 
 
 def get_author_info(author_name):
@@ -76,6 +109,7 @@ def get_author_info(author_name):
 
     if author_info:
         author_info._timestamp = now
+        author_info.safe_about = clean_string(author_info.about)
         _AUTHORS_CACHE[author_name] = author_info
     return author_info
 

cps/services/worker.py

@@ -266,3 +266,6 @@ class CalibreTask:
     def _handleSuccess(self):
         self.stat = STAT_FINISH_SUCCESS
         self.progress = 1
+
+    def __str__(self):
+        return self.name

cps/shelf.py

@@ -71,6 +71,14 @@ def add_to_shelf(shelf_id, book_id):
     else:
         maxOrder = maxOrder[0]
 
+    if not calibre_db.session.query(db.Books).filter(db.Books.id == book_id).one_or_none():
+        log.error("Invalid Book Id: %s. Could not be added to shelf %s", book_id, shelf.name)
+        if not xhr:
+            flash(_("%(book_id)s is a invalid Book Id. Could not be added to Shelf", book_id=book_id),
+                  category="error")
+            return redirect(url_for('web.index'))
+        return "%s is a invalid Book Id. Could not be added to Shelf" % book_id, 400
+
     shelf.books.append(ub.BookShelf(shelf=shelf.id, book_id=book_id, order=maxOrder + 1))
     shelf.last_modified = datetime.utcnow()
     try:
@@ -295,11 +303,14 @@ def check_shelf_edit_permissions(cur_shelf):
 
 
 def check_shelf_view_permissions(cur_shelf):
-    if cur_shelf.is_public:
-        return True
-    if current_user.is_anonymous or cur_shelf.user_id != current_user.id:
-        log.error("User is unauthorized to view non-public shelf: {}".format(cur_shelf.name))
-        return False
+    try:
+        if cur_shelf.is_public:
+            return True
+        if current_user.is_anonymous or cur_shelf.user_id != current_user.id:
+            log.error("User is unauthorized to view non-public shelf: {}".format(cur_shelf.name))
+            return False
+    except Exception as e:
+        log.error(e)
     return True
 
 

cps/static/css/caliBlur.css

@@ -3296,6 +3296,7 @@ div.btn-group[role=group][aria-label="Download, send to Kindle, reading"] .dropd
     left: 0 !important;
 }
 #add-to-shelves {
+  min-height: 48px;
   max-height: calc(100% - 120px);
   overflow-y: auto;
 }
@@ -4812,8 +4813,14 @@ body.advsearch:not(.blur) > div.container-fluid > div.row-fluid > div.col-sm-10
     z-index: 999999999999999999999999999999999999
 }
 
-.search #shelf-actions, body.login .home-btn {
-    display: none
+body.search #shelf-actions button#add-to-shelf {
+  height: 40px;
+}
+@media screen and (max-width: 767px) {
+  body.search .discover, body.advsearch .discover {
+    display: flex;
+    flex-direction: column;
+  }
 }
 
 body.read:not(.blur) a[href*=readbooks] {
@@ -5134,7 +5141,7 @@ body.login > div.navbar.navbar-default.navbar-static-top > div > div.navbar-head
     right: 5px
 }
 
-#shelf-actions > .btn-group.open, .downloadBtn.open, .profileDrop[aria-expanded=true] {
+body:not(.search) #shelf-actions > .btn-group.open, .downloadBtn.open, .profileDrop[aria-expanded=true] {
     pointer-events: none
 }
 
@@ -5151,7 +5158,7 @@ body.login > div.navbar.navbar-default.navbar-static-top > div > div.navbar-head
     color: var(--color-primary)
 }
 
-#shelf-actions, #shelf-actions > .btn-group, #shelf-actions > .btn-group > .empty-ul {
+body:not(.search) #shelf-actions, body:not(.search) #shelf-actions > .btn-group, body:not(.search) #shelf-actions > .btn-group > .empty-ul {
     pointer-events: none
 }
 
@@ -7279,6 +7286,11 @@ body.edituser.admin > div.container-fluid > div.row-fluid > div.col-sm-10 > div.
         float: right
     }
 
+    body.blur #main-nav + #scnd-nav .create-shelf, body.blur #main-nav + .col-sm-2 #scnd-nav .create-shelf {
+      float: none;
+      margin: 5px 0 10px -10px;
+    }
+
     #main-nav + #scnd-nav .nav-head.hidden-xs {
         display: list-item !important;
         width: 225px

cps/static/icon.svg

@@ -0,0 +1,5 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns="http://www.w3.org/2000/svg" version="1.1" width="140px" height="140px" style="shape-rendering:geometricPrecision; text-rendering:geometricPrecision; image-rendering:optimizeQuality; fill-rule:evenodd; clip-rule:evenodd" xmlns:xlink="http://www.w3.org/1999/xlink">
+<g><path style="opacity:1" fill="#45b29d" d="M 70.5,5.5 C 87.7691,3.12603 97.4358,10.4594 99.5,27.5C 95.637,46.6972 84.3037,59.1972 65.5,65C 60.9053,66.3929 56.2387,66.7262 51.5,66C 50.0692,65.5348 48.9025,64.7014 48,63.5C 47.3333,60.5 47.3333,57.5 48,54.5C 62.2513,56.0484 73.5846,50.715 82,38.5C 85.0332,33.8945 86.0332,28.8945 85,23.5C 83.0488,19.2854 79.7155,17.2854 75,17.5C 65.5257,19.0759 57.859,23.7425 52,31.5C 38.306,51.6368 33.9727,73.6368 39,97.5C 44.5639,116.532 56.7306,122.699 75.5,116C 80.6017,113.385 85.2684,110.218 89.5,106.5C 95.1927,108.891 96.6927,112.891 94,118.5C 78.4211,132.151 61.2544,134.651 42.5,126C 31.5182,117.21 25.3516,105.71 24,91.5C 20.9978,65.8515 27.3311,42.8515 43,22.5C 50.6154,14.1193 59.7821,8.45258 70.5,5.5 Z"/></g>
+</svg>

cps/static/js/caliBlur.js

@@ -81,56 +81,6 @@ if ($("body.book").length > 0) {
     $(".rating").insertBefore(".hr");
     $("#remove-from-shelves").insertAfter(".hr");
     $(description).appendTo(".bookinfo")
-    /* if book description is not in html format, Remove extra line breaks
-    Remove blank lines/unnecessary spaces, split by line break to array
-    Push array into .description div. If there is still a wall of text,
-    find sentences and split wall into groups of three sentence paragraphs.
-    If the book format is in html format, Keep html, but strip away inline
-    styles and empty elements */
-
-    // If text is sitting in div as text node
-    if ($(".comments:has(p)").length === 0) {
-        newdesc = description.text()
-            .replace(/^(?=\n)$|^\s*|\s*$|\n\n+/gm, "").split(/\n/);
-        $(".comments").empty();
-        $.each(newdesc, function (i, val) {
-            $("div.comments").append("<p>" + newdesc[i] + "</p>");
-        });
-        $(".comments").fadeIn(100);
-    }    //If still a wall of text create 3 sentence paragraphs.
-    if ($(".comments p").length === 1) {
-        if (description.context != undefined) {
-            newdesc = description.text()
-                .replace(/^(?=\n)$|^\s*|\s*$|\n\n+/gm, "").split(/\n/);
-        } else {
-            newdesc = description.text();
-        }
-        doc = nlp(newdesc.toString());
-        sentences = doc.map((m) => m.out("text"));
-        sentences[0] = sentences[0].replace(",", "");
-        $(".comments p").remove();
-        let size = 3;
-        let sentenceChunks = [];
-        for (var i = 0; i < sentences.length; i += size) {
-            sentenceChunks.push(sentences.slice(i, i + size));
-        }
-        let output = '';
-        $.each(sentenceChunks, function (i, val) {
-            let preOutput = '';
-            $.each(val, function (i, val) {
-                preOutput += val;
-            });
-            output += "<p>" + preOutput + "</p>";
-        });
-        $("div.comments").append(output);
-    } else {
-        $.each(description, function (i, val) {
-//      $( description[i].outerHTML ).appendTo( ".comments" );
-            $("div.comments :empty").remove();
-            $("div.comments ").attr("style", "");
-        });
-        $("div.comments").fadeIn(100);
-    }
 
     // Sexy blurred backgrounds
     cover = $(".cover img").attr("src");
@@ -369,6 +319,13 @@ $("div.comments").readmore({
 //     End of Global Work     //
 ///////////////////////////////
 
+// Search Results
+if($("body.search").length > 0) {
+  $('div[aria-label="Add to shelves"]').click(function () {
+    $("#add-to-shelves").toggle();
+  });
+}
+
 // Advanced Search Results
 if($("body.advsearch").length > 0) {
   $("#loader + .container-fluid")
@@ -503,6 +460,7 @@ if ($("body.shelf").length > 0) {
 // Rest of Tooltips
 $(".home-btn > a").attr({
     "data-toggle": "tooltip",
+    "href": $(".navbar-brand")[0].href,
     "title": $(document.body).attr("data-text"),    // Home
     "data-placement": "bottom"
 })

cps/static/js/details.js

@@ -40,6 +40,7 @@ $(".sendbtn-form").click(function() {
     $.ajax({
         method: 'post',
         url: $(this).data('href'),
+        data: {csrf_token: $("input[name='csrf_token']").val()},
         success: function (data) {
             handleResponse(data)
         }

cps/static/js/kthoom.js

@@ -71,7 +71,8 @@ var settings = {
     fitMode: kthoom.Key.B,
     theme: "light",
     direction: 0, // 0 = Left to Right, 1 = Right to Left
-	scrollbar: 1, // 0 = Hide Scrollbar, 1 = Show Scrollbar
+    nextPage: 0, // 0 = Reset to Top, 1 = Remember Position
+	scrollbar: 1, // 0 = Hide Scrollbar, 1 = Show Scrollbar	
     pageDisplay: 0 // 0 = Single Page, 1 = Long Strip
 };
 
@@ -131,8 +132,8 @@ var createURLFromArray = function(array, mimeType) {
     }
 
     if ((typeof URL !== "function" && typeof URL !== "object") ||
-      typeof URL.createObjectURL !== "function") {
-        throw "Browser support for Object URLs is missing";
+        typeof URL.createObjectURL !== "function") {
+            throw "Browser support for Object URLs is missing";
     }
 
     return URL.createObjectURL(blob);
@@ -177,12 +178,34 @@ kthoom.ImageFile = function(file) {
     }
 };
 
+function updateDirectionButtons(){
+    var left = 1;
+    var right = 1;
+    if (currentImage <= 0 ) {
+        if (settings.direction === 0) {
+            left = 0;
+        } else {
+            right = 0;
+        }
+    }
+    if ((currentImage + 1) >= Math.max(totalImages, imageFiles.length)) {
+        if (settings.direction === 0) {
+            right = 0;
+        } else {
+            left = 0;
+       }
+    }
+    left === 1 ? $("#left").show() : $("#left").hide();
+    right === 1 ? $("#right").show() : $("#right").hide();
+}
 function initProgressClick() {
     $("#progress").click(function(e) {
         var offset = $(this).offset();
         var x = e.pageX - offset.left;
         var rate = settings.direction === 0 ? x / $(this).width() : 1 - x / $(this).width();
         currentImage = Math.max(1, Math.ceil(rate * totalImages)) - 1;
+        updateDirectionButtons();
+        setBookmark();
         updatePage();
     });
 }
@@ -222,6 +245,7 @@ function loadFromArrayBuffer(ab) {
                                     
                                     // display first page if we haven't yet
                                     if (imageFiles.length === currentImage + 1) {
+                                        updateDirectionButtons();
                                         updatePage();
                                     }
                                 } else {
@@ -241,7 +265,7 @@ function scrollTocToActive() {
 
     // Mark the current page in the TOC
     $("#tocView a[data-page]")
-    // Remove the currently active thumbnail
+        // Remove the currently active thumbnail
         .removeClass("active")
         // Find the new one
         .filter("[data-page=" + (currentImage + 1) + "]")
@@ -409,6 +433,7 @@ function showLeftPage() {
     } else {
         showNextPage();
     }
+    setBookmark();
 }
 
 function showRightPage() {
@@ -417,6 +442,7 @@ function showRightPage() {
     } else {
         showPrevPage();
     }
+    setBookmark();
 }
 
 function showPrevPage() {
@@ -427,6 +453,7 @@ function showPrevPage() {
     } else {
         updatePage();
     }
+    updateDirectionButtons();
 }
 
 function showNextPage() {
@@ -437,6 +464,7 @@ function showNextPage() {
     } else {
         updatePage();
     }
+    updateDirectionButtons();
 }
 
 function scrollCurrentImageIntoView() {
@@ -621,11 +649,21 @@ function drawCanvas() {
     $("#mainContent").append(canvasElement);
 }
 
+function updateArrows() {
+    if ($('input[name="direction"]:checked').val() === "0") {
+        $("#prev_page_key").html("&larr;");
+        $("#next_page_key").html("&rarr;");
+    } else {
+        $("#prev_page_key").html("&rarr;");
+        $("#next_page_key").html("&larr;");
+    }
+};
+
 function init(filename) {
     var request = new XMLHttpRequest();
     request.open("GET", filename);
     request.responseType = "arraybuffer";
-    request.addEventListener("load", function() {
+    request.addEventListener("load", function () {
         if (request.status >= 200 && request.status < 300) {
             loadFromArrayBuffer(request.response);
         } else {
@@ -641,18 +679,18 @@ function init(filename) {
 
     $(document).keydown(keyHandler);
 
-    $(window).resize(function() {
+    $(window).resize(function () {
         updateScale();
     });
 
     // Open TOC menu
-    $("#slider").click(function() {
+    $("#slider").click(function () {
         $("#sidebar").toggleClass("open");
         $("#main").toggleClass("closed");
         $(this).toggleClass("icon-menu icon-right");
 
         // We need this in a timeout because if we call it during the CSS transition, IE11 shakes the page ¯\_(ツ)_/¯
-        setTimeout(function() {
+        setTimeout(function () {
             // Focus on the TOC or the main content area, depending on which is open
             $("#main:not(.closed) #mainContent, #sidebar.open #tocView").focus();
             scrollTocToActive();
@@ -660,12 +698,12 @@ function init(filename) {
     });
 
     // Open Settings modal
-    $("#setting").click(function() {
+    $("#setting").click(function () {
         $("#settings-modal").toggleClass("md-show");
     });
 
     // On Settings input change
-    $("#settings input").on("change", function() {
+    $("#settings input").on("change", function () {
         // Get either the checked boolean or the assigned value
         var value = this.type === "checkbox" ? this.checked : this.value;
 
@@ -674,39 +712,40 @@ function init(filename) {
 
         settings[this.name] = value;
 
-        if(["hflip", "vflip", "rotateTimes"].includes(this.name)) {
+        if (["hflip", "vflip", "rotateTimes"].includes(this.name)) {
             reloadImages();
-        } else if(this.name === "direction") {
+        } else if (this.name === "direction") {
+            updateDirectionButtons();
             return updateProgress();
         }
-        
+
         updatePage();
         updateScale();
     });
 
     // Close modal
-    $(".closer, .overlay").click(function() {
+    $(".closer, .overlay").click(function () {
         $(".md-show").removeClass("md-show");
-		$("#mainContent").focus(); // focus back on the main container so you use up/down keys without having to click on it
+        $("#mainContent").focus(); // focus back on the main container so you use up/down keys without having to click on it
     });
 
     // TOC thumbnail pagination
-    $("#thumbnails").on("click", "a", function() {
+    $("#thumbnails").on("click", "a", function () {
         currentImage = $(this).data("page") - 1;
         updatePage();
     });
 
     // Fullscreen mode
     if (typeof screenfull !== "undefined") {
-        $("#fullscreen").click(function() {
+        $("#fullscreen").click(function () {
             screenfull.toggle($("#container")[0]);
-			// Focus on main container so you can use up/down keys immediately after fullscreen
-			$("#mainContent").focus();
+            // Focus on main container so you can use up/down keys immediately after fullscreen
+            $("#mainContent").focus();
         });
 
         if (screenfull.raw) {
             var $button = $("#fullscreen");
-            document.addEventListener(screenfull.raw.fullscreenchange, function() {
+            document.addEventListener(screenfull.raw.fullscreenchange, function () {
                 screenfull.isFullscreen
                     ? $button.addClass("icon-resize-small").removeClass("icon-resize-full")
                     : $button.addClass("icon-resize-full").removeClass("icon-resize-small");
@@ -717,16 +756,16 @@ function init(filename) {
     // Focus the scrollable area so that keyboard scrolling work as expected
     $("#mainContent").focus();
 
-    $("#mainContent").swipe( {
-        swipeRight:function() {
+    $("#mainContent").swipe({
+        swipeRight: function () {
             showLeftPage();
         },
-        swipeLeft:function() {
+        swipeLeft: function () {
             showRightPage();
         },
     });
-    $(".mainImage").click(function(evt) {
-        // Firefox does not support offsetX/Y so we have to manually calculate
+    $(".mainImage").click(function (evt) {
+        // Firefox does not support offsetX/Y, so we have to manually calculate
         // where the user clicked in the image.
         var mainContentWidth = $("#mainContent").width();
         var mainContentHeight = $("#mainContent").height();
@@ -762,30 +801,38 @@ function init(filename) {
     });
 
     // Scrolling up/down will update current image if a new image is into view (for Long Strip Display)
-    $("#mainContent").scroll(function(){
+    $("#mainContent").scroll(function (){
         var scroll = $("#mainContent").scrollTop();
-        if(settings.pageDisplay === 0) {
+        var viewLength = 0;
+        $(".mainImage").each(function(){
+            viewLength += $(this).height();
+        });
+        if (settings.pageDisplay === 0) {
             // Don't trigger the scroll for Single Page
-        } else if(scroll > prevScrollPosition) {
+        } else if (scroll > prevScrollPosition) {
             //Scroll Down
-            if(currentImage + 1 < imageFiles.length) {
-                if(currentImageOffset(currentImage + 1) <= 1) {
-                    currentImage++;
+            if (currentImage + 1 < imageFiles.length) {
+                if (currentImageOffset(currentImage + 1) <= 1) {
+                    currentImage = Math.floor((imageFiles.length) / (viewLength-viewLength/(imageFiles.length)) * scroll, 0);
+                    if ( currentImage >= imageFiles.length) {
+                        currentImage = imageFiles.length - 1;
+                    }
+                    console.log(currentImage);
                     scrollTocToActive();
                     updateProgress();
                 }
             }
         } else {
             //Scroll Up
-            if(currentImage - 1 > -1 ) {
-                if(currentImageOffset(currentImage - 1) >= 0) {
-                    currentImage--;
+            if (currentImage - 1 > -1) {
+                if (currentImageOffset(currentImage - 1) >= 0) {
+                    currentImage = Math.floor((imageFiles.length) / (viewLength-viewLength/(imageFiles.length)) * scroll, 0);
+                    console.log(currentImage);
                     scrollTocToActive();
                     updateProgress();
                 }
             }
         }
-
         // Update scroll position
         prevScrollPosition = scroll;
     });
@@ -794,3 +841,31 @@ function init(filename) {
 function currentImageOffset(imageIndex) {
     return $(".mainImage").eq(imageIndex).offset().top - $("#mainContent").position().top
 }
+
+function setBookmark() {
+  // get csrf_token
+    let csrf_token = $("input[name='csrf_token']").val();
+    //This sends a bookmark update to calibreweb.
+    $.ajax(calibre.bookmarkUrl, {
+        method: "post",
+        data: {
+            csrf_token: csrf_token,
+            bookmark: currentImage
+        }
+    }).fail(function (xhr, status, error) {
+        console.error(error);
+    });
+}
+
+$(function() {
+    $('input[name="direction"]').change(function () {
+        updateArrows();
+    });
+
+    $('#left').click(function () {
+        showLeftPage();
+    });
+    $('#right').click(function () {
+        showRightPage();
+    });
+});

cps/static/js/libs/bootstrap-datepicker/locales/bootstrap-datepicker.pt.min.js

@@ -0,0 +1 @@
+!function(a){a.fn.datepicker.dates.pt={days:["Domingo","Segunda","Terça","Quarta","Quinta","Sexta","Sábado"],daysShort:["Dom","Seg","Ter","Qua","Qui","Sex","Sáb"],daysMin:["Do","Se","Te","Qu","Qu","Se","Sa"],months:["Janeiro","Fevereiro","Março","Abril","Maio","Junho","Julho","Agosto","Setembro","Outubro","Novembro","Dezembro"],monthsShort:["Jan","Fev","Mar","Abr","Mai","Jun","Jul","Ago","Set","Out","Nov","Dez"],today:"Hoje",monthsTitle:"Meses",clear:"Limpar",format:"dd/mm/yyyy"}}(jQuery);

cps/static/js/libs/bootstrap-datepicker/locales/bootstrap-datepicker.sk.min.js

@@ -0,0 +1 @@
+!function(a){a.fn.datepicker.dates.sk={days:["Nedeľa","Pondelok","Utorok","Streda","Štvrtok","Piatok","Sobota"],daysShort:["Ned","Pon","Uto","Str","Štv","Pia","Sob"],daysMin:["Ne","Po","Ut","St","Št","Pia","So"],months:["Január","Február","Marec","Apríl","Máj","Jún","Júl","August","September","Október","November","December"],monthsShort:["Jan","Feb","Mar","Apr","Máj","Jún","Júl","Aug","Sep","Okt","Nov","Dec"],today:"Dnes",clear:"Vymazať",weekStart:1,format:"d.m.yyyy"}}(jQuery);

cps/static/js/libs/pwstrength/locales/de.json

@@ -9,6 +9,7 @@
     "wordSequences": "Das Passwort enthält Buchstabensequenzen",
     "wordLowercase": "Bitte mindestens einen Kleinbuchstaben verwenden",
     "wordUppercase": "Bitte mindestens einen Großbuchstaben verwenden",
+    "word": "Bitte mindestens einen Buchstaben verwenden",
     "wordOneNumber": "Bitte mindestens eine Ziffern verwenden",
     "wordOneSpecialChar": "Bitte mindestens ein Sonderzeichen verwenden",
     "errorList": "Fehler:",

cps/static/js/libs/pwstrength/locales/en.json

@@ -8,6 +8,7 @@
     "wordRepetitions": "Too many repetitions",
     "wordSequences": "Your password contains sequences",
     "wordLowercase": "Use at least one lowercase character",
+    "word": "Use at least one character",
     "wordUppercase": "Use at least one uppercase character",
     "wordOneNumber": "Use at least one number",
     "wordOneSpecialChar": "Use at least one special character",

cps/static/js/libs/pwstrength/pwstrength-bootstrap.js

@@ -144,13 +144,13 @@ try {
 
     validation.wordTwoCharacterClasses = function(options, word, score) {
         var specialCharRE = new RegExp(
-            '(.' + options.rules.specialCharClass + ')'
+            '(.' + options.rules.specialCharClass + ')', 'u'
         );
 
         if (
-            word.match(/([a-z].*[A-Z])|([A-Z].*[a-z])/) ||
-            (word.match(/([a-zA-Z])/) && word.match(/([0-9])/)) ||
-            (word.match(specialCharRE) && word.match(/[a-zA-Z0-9_]/))
+            word.match(/(\p{Ll}.*\p{Lu})|(\p{Lu}.*\p{Ll})/u) ||
+            (word.match(/(\p{Letter})/u) && word.match(/([0-9])/)) ||
+            (word.match(specialCharRE) && word.match(/[\p{Letter}0-9_]/u))
         ) {
             return score;
         }
@@ -202,11 +202,15 @@ try {
     };
 
     validation.wordLowercase = function(options, word, score) {
-        return word.match(/[a-z]/) && score;
+        return word.match(/\p{Ll}/u) && score;
     };
 
     validation.wordUppercase = function(options, word, score) {
-        return word.match(/[A-Z]/) && score;
+        return word.match(/\p{Lu}/u) && score;
+    };
+
+    validation.word = function(options, word, score) {
+        return word.match(/\p{Letter}/u) && score;
     };
 
     validation.wordOneNumber = function(options, word, score) {
@@ -218,7 +222,7 @@ try {
     };
 
     validation.wordOneSpecialChar = function(options, word, score) {
-        var specialCharRE = new RegExp(options.rules.specialCharClass);
+        var specialCharRE = new RegExp(options.rules.specialCharClass, 'u');
         return word.match(specialCharRE) && score;
     };
 
@@ -228,27 +232,27 @@ try {
                 options.rules.specialCharClass +
                 '.*' +
                 options.rules.specialCharClass +
-                ')'
+                ')', 'u'
         );
 
         return word.match(twoSpecialCharRE) && score;
     };
 
     validation.wordUpperLowerCombo = function(options, word, score) {
-        return word.match(/([a-z].*[A-Z])|([A-Z].*[a-z])/) && score;
+        return word.match(/(\p{Ll}.*\p{Lu})|(\p{Lu}.*\p{Ll})/u) && score;
     };
 
     validation.wordLetterNumberCombo = function(options, word, score) {
-        return word.match(/([a-zA-Z])/) && word.match(/([0-9])/) && score;
+        return word.match(/([\p{Letter}])/u) && word.match(/([0-9])/) && score;
     };
 
     validation.wordLetterNumberCharCombo = function(options, word, score) {
         var letterNumberCharComboRE = new RegExp(
-            '([a-zA-Z0-9].*' +
+            '([\p{Letter}0-9].*' +
                 options.rules.specialCharClass +
                 ')|(' +
                 options.rules.specialCharClass +
-                '.*[a-zA-Z0-9])'
+                '.*[\p{Letter}0-9])', 'u'
         );
 
         return word.match(letterNumberCharComboRE) && score;
@@ -341,6 +345,7 @@ defaultOptions.rules.scores = {
     wordTwoCharacterClasses: 2,
     wordRepetitions: -25,
     wordLowercase: 1,
+    word: 1,
     wordUppercase: 3,
     wordOneNumber: 3,
     wordThreeNumbers: 5,
@@ -361,6 +366,7 @@ defaultOptions.rules.activated = {
     wordTwoCharacterClasses: true,
     wordRepetitions: true,
     wordLowercase: true,
+    word: true,
     wordUppercase: true,
     wordOneNumber: true,
     wordThreeNumbers: true,
@@ -372,7 +378,7 @@ defaultOptions.rules.activated = {
     wordIsACommonPassword: true
 };
 defaultOptions.rules.raisePower = 1.4;
-defaultOptions.rules.specialCharClass = "(?=.*?[^A-Za-z\s0-9])"; //'[!,@,#,$,%,^,&,*,?,_,~]';
+defaultOptions.rules.specialCharClass = "(?=.*?[^\\p{Letter}\\s0-9])"; //'[!,@,#,$,%,^,&,*,?,_,~]';
 // List taken from https://github.com/danielmiessler/SecLists (MIT License)
 defaultOptions.rules.commonPasswords = [
     '123456',

cps/static/js/libs/tinymce/langs/pt.js

@@ -0,0 +1,462 @@
+tinymce.addI18n('pt_PT',{
+"Redo": "Refazer",
+"Undo": "Anular",
+"Cut": "Cortar",
+"Copy": "Copiar",
+"Paste": "Colar",
+"Select all": "Selecionar tudo",
+"New document": "Novo documento",
+"Ok": "Ok",
+"Cancel": "Cancelar",
+"Visual aids": "Ajuda visual",
+"Bold": "Negrito",
+"Italic": "It\u00e1lico",
+"Underline": "Sublinhado",
+"Strikethrough": "Rasurado",
+"Superscript": "Superior \u00e0 linha",
+"Subscript": "Inferior \u00e0 linha",
+"Clear formatting": "Limpar formata\u00e7\u00e3o",
+"Align left": "Alinhar \u00e0 esquerda",
+"Align center": "Alinhar ao centro",
+"Align right": "Alinhar \u00e0 direita",
+"Justify": "Justificar",
+"Bullet list": "Lista com marcas",
+"Numbered list": "Lista numerada",
+"Decrease indent": "Diminuir avan\u00e7o",
+"Increase indent": "Aumentar avan\u00e7o",
+"Close": "Fechar",
+"Formats": "Formatos",
+"Your browser doesn't support direct access to the clipboard. Please use the Ctrl+X\/C\/V keyboard shortcuts instead.": "O seu navegador n\u00e3o suporta acesso direto \u00e0 \u00e1rea de transfer\u00eancia. Por favor, use os atalhos Ctrl+X\/C\/V do seu teclado.",
+"Headers": "Cabe\u00e7alhos",
+"Header 1": "Cabe\u00e7alho 1",
+"Header 2": "Cabe\u00e7alho 2",
+"Header 3": "Cabe\u00e7alho 3",
+"Header 4": "Cabe\u00e7alho 4",
+"Header 5": "Cabe\u00e7alho 5",
+"Header 6": "Cabe\u00e7alho 6",
+"Headings": "T\u00edtulos",
+"Heading 1": "T\u00edtulo 1",
+"Heading 2": "T\u00edtulo 2",
+"Heading 3": "T\u00edtulo 3",
+"Heading 4": "T\u00edtulo 4",
+"Heading 5": "T\u00edtulo 5",
+"Heading 6": "T\u00edtulo 6",
+"Preformatted": "Pr\u00e9-formatado",
+"Div": "Div",
+"Pre": "Pre",
+"Code": "C\u00f3digo",
+"Paragraph": "Par\u00e1grafo",
+"Blockquote": "Blockquote",
+"Inline": "Inline",
+"Blocks": "Blocos",
+"Paste is now in plain text mode. Contents will now be pasted as plain text until you toggle this option off.": "O comando colar est\u00e1 em modo de texto simples. O conte\u00fado ser\u00e1 colado como texto simples at\u00e9 desativar esta op\u00e7\u00e3o.",
+"Fonts": "Tipos de letra",
+"Font Sizes": "Tamanhos dos tipos de letra",
+"Class": "Classe",
+"Browse for an image": "Procurar uma imagem",
+"OR": "OU",
+"Drop an image here": "Largar aqui uma imagem",
+"Upload": "Carregar",
+"Block": "Bloco",
+"Align": "Alinhar",
+"Default": "Padr\u00e3o",
+"Circle": "C\u00edrculo",
+"Disc": "Disco",
+"Square": "Quadrado",
+"Lower Alpha": "a. b. c. ...",
+"Lower Greek": "\\u03b1. \\u03b2. \\u03b3. ...",
+"Lower Roman": "i. ii. iii. ...",
+"Upper Alpha": "A. B. C. ...",
+"Upper Roman": "I. II. III. ...",
+"Anchor...": "\u00c2ncora...",
+"Name": "Nome",
+"Id": "ID",
+"Id should start with a letter, followed only by letters, numbers, dashes, dots, colons or underscores.": "O ID deve come\u00e7ar com uma letra, seguido apenas por letras, n\u00fameros, pontos, dois pontos, tra\u00e7os ou sobtra\u00e7os.",
+"You have unsaved changes are you sure you want to navigate away?": "Existem altera\u00e7\u00f5es que ainda n\u00e3o foram guardadas. Tem a certeza que pretende sair?",
+"Restore last draft": "Restaurar o \u00faltimo rascunho",
+"Special character...": "Car\u00e1ter especial...",
+"Source code": "C\u00f3digo fonte",
+"Insert\/Edit code sample": "Inserir\/editar amostra de c\u00f3digo",
+"Language": "Idioma",
+"Code sample...": "Amostra de c\u00f3digo...",
+"Color Picker": "Seletor de cores",
+"R": "R",
+"G": "G",
+"B": "B",
+"Left to right": "Da esquerda para a direita",
+"Right to left": "Da direita para a esquerda",
+"Emoticons": "Emo\u00e7\u00f5es",
+"Emoticons...": "\u00cdcones expressivos...",
+"Metadata and Document Properties": "Metadados e propriedades do documento",
+"Title": "T\u00edtulo",
+"Keywords": "Palavras-chave",
+"Description": "Descri\u00e7\u00e3o",
+"Robots": "Rob\u00f4s",
+"Author": "Autor",
+"Encoding": "Codifica\u00e7\u00e3o",
+"Fullscreen": "Ecr\u00e3 completo",
+"Action": "A\u00e7\u00e3o",
+"Shortcut": "Atalho",
+"Help": "Ajuda",
+"Address": "Endere\u00e7o",
+"Focus to menubar": "Foco na barra de menu",
+"Focus to toolbar": "Foco na barra de ferramentas",
+"Focus to element path": "Foco no caminho do elemento",
+"Focus to contextual toolbar": "Foco na barra de contexto",
+"Insert link (if link plugin activated)": "Inserir hiperliga\u00e7\u00e3o (se o plugin de liga\u00e7\u00f5es estiver ativado)",
+"Save (if save plugin activated)": "Guardar (se o plugin de guardar estiver ativado)",
+"Find (if searchreplace plugin activated)": "Pesquisar (se o plugin pesquisar e substituir estiver ativado)",
+"Plugins installed ({0}):": "Plugins instalados ({0}):",
+"Premium plugins:": "Plugins comerciais:",
+"Learn more...": "Saiba mais...",
+"You are using {0}": "Est\u00e1 a usar {0}",
+"Plugins": "Plugins",
+"Handy Shortcuts": "Atalhos \u00fateis",
+"Horizontal line": "Linha horizontal",
+"Insert\/edit image": "Inserir\/editar imagem",
+"Alternative description": "Descri\u00e7\u00e3o alternativa",
+"Accessibility": "Acessibilidade",
+"Image is decorative": "Imagem \u00e9 decorativa",
+"Source": "Localiza\u00e7\u00e3o",
+"Dimensions": "Dimens\u00f5es",
+"Constrain proportions": "Manter propor\u00e7\u00f5es",
+"General": "Geral",
+"Advanced": "Avan\u00e7ado",
+"Style": "Estilo",
+"Vertical space": "Espa\u00e7amento vertical",
+"Horizontal space": "Espa\u00e7amento horizontal",
+"Border": "Contorno",
+"Insert image": "Inserir imagem",
+"Image...": "Imagem...",
+"Image list": "Lista de imagens",
+"Rotate counterclockwise": "Rota\u00e7\u00e3o anti-hor\u00e1ria",
+"Rotate clockwise": "Rota\u00e7\u00e3o hor\u00e1ria",
+"Flip vertically": "Inverter verticalmente",
+"Flip horizontally": "Inverter horizontalmente",
+"Edit image": "Editar imagem",
+"Image options": "Op\u00e7\u00f5es de imagem",
+"Zoom in": "Mais zoom",
+"Zoom out": "Menos zoom",
+"Crop": "Recortar",
+"Resize": "Redimensionar",
+"Orientation": "Orienta\u00e7\u00e3o",
+"Brightness": "Brilho",
+"Sharpen": "Mais nitidez",
+"Contrast": "Contraste",
+"Color levels": "N\u00edveis de cor",
+"Gamma": "Gama",
+"Invert": "Inverter",
+"Apply": "Aplicar",
+"Back": "Voltar",
+"Insert date\/time": "Inserir data\/hora",
+"Date\/time": "Data\/hora",
+"Insert\/edit link": "Inserir\/editar liga\u00e7\u00e3o",
+"Text to display": "Texto a exibir",
+"Url": "URL",
+"Open link in...": "Abrir liga\u00e7\u00e3o em...",
+"Current window": "Janela atual",
+"None": "Nenhum",
+"New window": "Nova janela",
+"Open link": "Abrir liga\u00e7\u00e3o",
+"Remove link": "Remover liga\u00e7\u00e3o",
+"Anchors": "\u00c2ncora",
+"Link...": "Liga\u00e7\u00e3o...",
+"Paste or type a link": "Copiar ou escrever uma hiperliga\u00e7\u00e3o",
+"The URL you entered seems to be an email address. Do you want to add the required mailto: prefix?": "O URL que indicou parece ser um endere\u00e7o de email. Quer adicionar o prefixo mailto: tal como necess\u00e1rio?",
+"The URL you entered seems to be an external link. Do you want to add the required http:\/\/ prefix?": "O URL que indicou parece ser um endere\u00e7o web. Quer adicionar o prefixo http:\/\/ tal como necess\u00e1rio?",
+"The URL you entered seems to be an external link. Do you want to add the required https:\/\/ prefix?": "O URL que introduziu parece ser uma liga\u00e7\u00e3o externa. Deseja adicionar-lhe o prefixo https:\/\/ ?",
+"Link list": "Lista de liga\u00e7\u00f5es",
+"Insert video": "Inserir v\u00eddeo",
+"Insert\/edit video": "Inserir\/editar v\u00eddeo",
+"Insert\/edit media": "Inserir\/editar media",
+"Alternative source": "Localiza\u00e7\u00e3o alternativa",
+"Alternative source URL": "URL da origem alternativa",
+"Media poster (Image URL)": "Publicador de media (URL da imagem)",
+"Paste your embed code below:": "Colar c\u00f3digo para embeber:",
+"Embed": "Embeber",
+"Media...": "Media...",
+"Nonbreaking space": "Espa\u00e7o n\u00e3o quebr\u00e1vel",
+"Page break": "Quebra de p\u00e1gina",
+"Paste as text": "Colar como texto",
+"Preview": "Pr\u00e9-visualizar",
+"Print...": "Imprimir...",
+"Save": "Guardar",
+"Find": "Pesquisar",
+"Replace with": "Substituir por",
+"Replace": "Substituir",
+"Replace all": "Substituir tudo",
+"Previous": "Anterior",
+"Next": "Pr\u00f3ximo",
+"Find and Replace": "Pesquisar e substituir",
+"Find and replace...": "Localizar e substituir...",
+"Could not find the specified string.": "N\u00e3o foi poss\u00edvel localizar o termo especificado.",
+"Match case": "Diferenciar mai\u00fasculas e min\u00fasculas",
+"Find whole words only": "Localizar apenas palavras inteiras",
+"Find in selection": "Pesquisar na selec\u00e7\u00e3o",
+"Spellcheck": "Corretor ortogr\u00e1fico",
+"Spellcheck Language": "Idioma de verifica\u00e7\u00e3o lingu\u00edstica",
+"No misspellings found.": "N\u00e3o foram encontrados erros ortogr\u00e1ficos.",
+"Ignore": "Ignorar",
+"Ignore all": "Ignorar tudo",
+"Finish": "Concluir",
+"Add to Dictionary": "Adicionar ao dicion\u00e1rio",
+"Insert table": "Inserir tabela",
+"Table properties": "Propriedades da tabela",
+"Delete table": "Eliminar tabela",
+"Cell": "C\u00e9lula",
+"Row": "Linha",
+"Column": "Coluna",
+"Cell properties": "Propriedades da c\u00e9lula",
+"Merge cells": "Unir c\u00e9lulas",
+"Split cell": "Dividir c\u00e9lula",
+"Insert row before": "Inserir linha antes",
+"Insert row after": "Inserir linha depois",
+"Delete row": "Eliminar linha",
+"Row properties": "Propriedades da linha",
+"Cut row": "Cortar linha",
+"Copy row": "Copiar linha",
+"Paste row before": "Colar linha antes",
+"Paste row after": "Colar linha depois",
+"Insert column before": "Inserir coluna antes",
+"Insert column after": "Inserir coluna depois",
+"Delete column": "Eliminar coluna",
+"Cols": "Colunas",
+"Rows": "Linhas",
+"Width": "Largura",
+"Height": "Altura",
+"Cell spacing": "Espa\u00e7amento entre c\u00e9lulas",
+"Cell padding": "Espa\u00e7amento interno da c\u00e9lula",
+"Caption": "Legenda",
+"Show caption": "Mostrar legenda",
+"Left": "Esquerda",
+"Center": "Centro",
+"Right": "Direita",
+"Cell type": "Tipo de c\u00e9lula",
+"Scope": "Escopo",
+"Alignment": "Alinhamento",
+"H Align": "Alinhamento H",
+"V Align": "Alinhamento V",
+"Top": "Superior",
+"Middle": "Meio",
+"Bottom": "Inferior",
+"Header cell": "C\u00e9lula de cabe\u00e7alho",
+"Row group": "Agrupar linha",
+"Column group": "Agrupar coluna",
+"Row type": "Tipo de linha",
+"Header": "Cabe\u00e7alho",
+"Body": "Corpo",
+"Footer": "Rodap\u00e9",
+"Border color": "Cor de contorno",
+"Insert template...": "Inserir modelo...",
+"Templates": "Modelos",
+"Template": "Tema",
+"Text color": "Cor do texto",
+"Background color": "Cor de fundo",
+"Custom...": "Personalizada...",
+"Custom color": "Cor personalizada",
+"No color": "Sem cor",
+"Remove color": "Remover cor",
+"Table of Contents": "\u00cdndice",
+"Show blocks": "Mostrar blocos",
+"Show invisible characters": "Mostrar caracteres invis\u00edveis",
+"Word count": "Contagem de palavras",
+"Count": "Contagem",
+"Document": "Documento",
+"Selection": "Sele\u00e7\u00e3o",
+"Words": "Palavras",
+"Words: {0}": "Palavras: {0}",
+"{0} words": "{0} palavras",
+"File": "Ficheiro",
+"Edit": "Editar",
+"Insert": "Inserir",
+"View": "Ver",
+"Format": "Formatar",
+"Table": "Tabela",
+"Tools": "Ferramentas",
+"Powered by {0}": "Criado em {0}",
+"Rich Text Area. Press ALT-F9 for menu. Press ALT-F10 for toolbar. Press ALT-0 for help": "Caixa de texto formatado. Pressione ALT-F9 para exibir o menu. Pressione ALT-F10 para exibir a barra de ferramentas. Pressione ALT-0 para exibir a ajuda",
+"Image title": "T\u00edtulo da imagem",
+"Border width": "Largura do limite",
+"Border style": "Estilo do limite",
+"Error": "Erro",
+"Warn": "Aviso",
+"Valid": "V\u00e1lido",
+"To open the popup, press Shift+Enter": "Para abrir o pop-up, prima Shift+Enter",
+"Rich Text Area. Press ALT-0 for help.": "\u00c1rea de texto formatado. Prima ALT-0 para exibir a ajuda.",
+"System Font": "Tipo de letra do sistema",
+"Failed to upload image: {0}": "Falha ao carregar imagem: {0}",
+"Failed to load plugin: {0} from url {1}": "Falha ao carregar plugin: {0} do URL {1}",
+"Failed to load plugin url: {0}": "Falha ao carregar o URL do plugin: {0}",
+"Failed to initialize plugin: {0}": "Falha ao inicializar plugin: {0}",
+"example": "exemplo",
+"Search": "Pesquisar",
+"All": "Tudo",
+"Currency": "Moeda",
+"Text": "Texto",
+"Quotations": "Aspas",
+"Mathematical": "Matem\u00e1tico",
+"Extended Latin": "Carateres latinos estendidos",
+"Symbols": "S\u00edmbolos",
+"Arrows": "Setas",
+"User Defined": "Definido pelo utilizador",
+"dollar sign": "cifr\u00e3o",
+"currency sign": "sinal monet\u00e1rio",
+"euro-currency sign": "sinal monet\u00e1rio do euro",
+"colon sign": "sinal de dois pontos",
+"cruzeiro sign": "sinal de cruzeiro",
+"french franc sign": "sinal de franco franc\u00eas",
+"lira sign": "sinal de lira",
+"mill sign": "sinal de por mil",
+"naira sign": "sinal de naira",
+"peseta sign": "sinal de peseta",
+"rupee sign": "sinal de r\u00fapia",
+"won sign": "sinal de won",
+"new sheqel sign": "sinal de novo sheqel",
+"dong sign": "sinal de dong",
+"kip sign": "sinal kip",
+"tugrik sign": "sinal tugrik",
+"drachma sign": "sinal drachma",
+"german penny symbol": "sinal de penny alem\u00e3o",
+"peso sign": "sinal de peso",
+"guarani sign": "sinal de guarani",
+"austral sign": "sinal de austral",
+"hryvnia sign": "sinal hryvnia",
+"cedi sign": "sinal de cedi",
+"livre tournois sign": "sinal de libra de tours",
+"spesmilo sign": "sinal de spesmilo",
+"tenge sign": "sinal de tengue",
+"indian rupee sign": "sinal de rupia indiana",
+"turkish lira sign": "sinal de lira turca",
+"nordic mark sign": "sinal de marca n\u00f3rdica",
+"manat sign": "sinal manat",
+"ruble sign": "sinal de rublo",
+"yen character": "sinal de iene",
+"yuan character": "sinal de iuane",
+"yuan character, in hong kong and taiwan": "sinal de iuane, em Hong Kong e Taiwan",
+"yen\/yuan character variant one": "variante um de sinal de iene\/iuane",
+"Loading emoticons...": "A carregar \u00edcones expressivos...",
+"Could not load emoticons": "N\u00e3o foi poss\u00edvel carregar \u00edcones expressivos",
+"People": "Pessoas",
+"Animals and Nature": "Animais e natureza",
+"Food and Drink": "Comida e bebida",
+"Activity": "Atividade",
+"Travel and Places": "Viagens e lugares",
+"Objects": "Objetos",
+"Flags": "Bandeiras",
+"Characters": "Carateres",
+"Characters (no spaces)": "Carateres (sem espa\u00e7os)",
+"{0} characters": "{0} carateres",
+"Error: Form submit field collision.": "Erro: conflito no campo de submiss\u00e3o de formul\u00e1rio.",
+"Error: No form element found.": "Erro: nenhum elemento de formul\u00e1rio encontrado.",
+"Update": "Atualizar",
+"Color swatch": "Cole\u00e7\u00e3o de cores",
+"Turquoise": "Turquesa",
+"Green": "Verde",
+"Blue": "Azul",
+"Purple": "P\u00farpura",
+"Navy Blue": "Azul-atl\u00e2ntico",
+"Dark Turquoise": "Turquesa escuro",
+"Dark Green": "Verde escuro",
+"Medium Blue": "Azul interm\u00e9dio",
+"Medium Purple": "P\u00farpura interm\u00e9dio",
+"Midnight Blue": "Azul muito escuro",
+"Yellow": "Amarelo",
+"Orange": "Laranja",
+"Red": "Vermelho",
+"Light Gray": "Cinzento claro",
+"Gray": "Cinzento",
+"Dark Yellow": "Amarelo escuro",
+"Dark Orange": "Laranja escuro",
+"Dark Red": "Vermelho escuro",
+"Medium Gray": "Cinzento m\u00e9dio",
+"Dark Gray": "Cinzento escuro",
+"Light Green": "Verde claro",
+"Light Yellow": "Amarelo claro",
+"Light Red": "Vermelho claro",
+"Light Purple": "P\u00farpura claro",
+"Light Blue": "Azul claro",
+"Dark Purple": "P\u00farpura escuro",
+"Dark Blue": "Azul escuro",
+"Black": "Preto",
+"White": "Branco",
+"Switch to or from fullscreen mode": "Entrar ou sair do modo de ecr\u00e3 inteiro",
+"Open help dialog": "Abrir caixa de di\u00e1logo Ajuda",
+"history": "hist\u00f3rico",
+"styles": "estilos",
+"formatting": "formata\u00e7\u00e3o",
+"alignment": "alinhamento",
+"indentation": "avan\u00e7o",
+"Font": "Tipo de letra",
+"Size": "Tamanho",
+"More...": "Mais...",
+"Select...": "Selecionar...",
+"Preferences": "Prefer\u00eancias",
+"Yes": "Sim",
+"No": "N\u00e3o",
+"Keyboard Navigation": "Navega\u00e7\u00e3o com teclado",
+"Version": "Vers\u00e3o",
+"Code view": "Vista do c\u00f3digo-fonte",
+"Open popup menu for split buttons": "Abrir o menu popup para bot\u00f5es divididos",
+"List Properties": "Propriedades da lista",
+"List properties...": "Propriedades da lista\u2026",
+"Start list at number": "Come\u00e7ar a lista pelo n\u00famero",
+"Line height": "Altura da linha",
+"comments": "coment\u00e1rios",
+"Format Painter": "Pincel de formata\u00e7\u00e3o",
+"Insert\/edit iframe": "Inserir\/editar iframe",
+"Capitalization": "Capitaliza\u00e7\u00e3o",
+"lowercase": "min\u00fasculas",
+"UPPERCASE": "MAI\u00daSCULAS",
+"Title Case": "Iniciais mai\u00fasculas",
+"permanent pen": "caneta permanente",
+"Permanent Pen Properties": "Propriedades da Caneta Permanente",
+"Permanent pen properties...": "Propriedades da caneta permanente...",
+"case change": "mudan\u00e7a de capitaliza\u00e7\u00e3o",
+"page embed": "incorporar p\u00e1gina",
+"Advanced sort...": "Ordena\u00e7\u00e3o avan\u00e7ada\u2026",
+"Advanced Sort": "Ordena\u00e7\u00e3o avan\u00e7ada",
+"Sort table by column ascending": "Ordenar tabela por coluna ascendente",
+"Sort table by column descending": "Ordenar tabela por coluna descendente",
+"Sort": "Ordenar",
+"Order": "Ordem",
+"Sort by": "Ordenar por",
+"Ascending": "Ascendente",
+"Descending": "Descendente",
+"Column {0}": "Coluna {0}",
+"Row {0}": "Linha {0}",
+"Spellcheck...": "Verifica\u00e7\u00e3o ortogr\u00e1fica...",
+"Misspelled word": "Palavra mal escrita",
+"Suggestions": "Sugest\u00f5es",
+"Change": "Alterar",
+"Finding word suggestions": "Encontrar sugest\u00f5es de palavras",
+"Success": "Sucesso",
+"Repair": "Reparar",
+"Issue {0} of {1}": "Problema {0} de {1}",
+"Images must be marked as decorative or have an alternative text description": "As imagens devem ser marcadas como decorativas ou ter uma descri\u00e7\u00e3o textual alternativa",
+"Images must have an alternative text description. Decorative images are not allowed.": "As imagens devem ter uma descri\u00e7\u00e3o textual alternativa. N\u00e3o s\u00e3o permitidas imagens meramente decorativas.",
+"Or provide alternative text:": "Ou forne\u00e7a um texto alternativo:",
+"Make image decorative:": "Marque a imagem como decorativa:",
+"ID attribute must be unique": "O atributo ID tem de ser \u00fanico",
+"Make ID unique": "Tornar o ID \u00fanico",
+"Keep this ID and remove all others": "Mantenha este ID e remova todos os outros",
+"Remove this ID": "Remover este ID",
+"Remove all IDs": "Remover todos os IDs",
+"Checklist": "Lista de verifica\u00e7\u00e3o",
+"Anchor": "\u00c2ncora",
+"Special character": "Car\u00e1cter especial",
+"Code sample": "Amostra de c\u00f3digo",
+"Color": "Cor",
+"Document properties": "Propriedades do documento",
+"Image description": "Descri\u00e7\u00e3o da imagem",
+"Image": "Imagem",
+"Insert link": "Inserir liga\u00e7\u00e3o",
+"Target": "Alvo",
+"Link": "Liga\u00e7\u00e3o",
+"Poster": "Autor",
+"Media": "Media",
+"Print": "Imprimir",
+"Prev": "Anterior",
+"Find and replace": "Pesquisar e substituir",
+"Whole words": "Palavras completas",
+"Insert template": "Inserir modelo"
+});

cps/static/js/logviewer.js

@@ -36,7 +36,7 @@ function init(logType) {
     d.innerHTML = "loading ...";
 
     $.ajax({
-        url: getPath() + "/../../ajax/log/" + logType,
+        url: getPath() + "/ajax/log/" + logType,
         datatype: "text",
         cache: false
     })

cps/static/js/main.js

@@ -20,7 +20,7 @@ function getPath() {
     return jsFileLocation.substr(0, jsFileLocation.search("/static/js/libs/jquery.min.js"));  // the js folder path
 }
 
-function postButton(event, action){
+function postButton(event, action, location=""){
     event.preventDefault();
     var newForm = jQuery('<form>', {
         "action": action,
@@ -30,7 +30,14 @@ function postButton(event, action){
         'name': 'csrf_token',
         'value': $("input[name=\'csrf_token\']").val(),
         'type': 'hidden'
-    })).appendTo('body');
+    })).appendTo('body')
+    if(location !== "") {
+        newForm.append(jQuery('<input>', {
+            'name': 'location',
+            'value': location,
+            'type': 'hidden'
+        })).appendTo('body');
+    }
     newForm.submit();
 }
 
@@ -212,17 +219,20 @@ $("#delete_confirm").click(function(event) {
                             $( ".navbar" ).after( '<div class="row-fluid text-center" >' +
                                 '<div id="flash_'+item.type+'" class="alert alert-'+item.type+'">'+item.message+'</div>' +
                                 '</div>');
-
                         }
                     });
                     $("#books-table").bootstrapTable("refresh");
                 }
             });
         } else {
-            postButton(event, getPath() + "/delete/" + deleteId);
+            var loc = sessionStorage.getItem("back");
+            if (!loc) {
+                loc = $(this).data("back");
+            }
+            sessionStorage.removeItem("back");
+            postButton(event, getPath() + "/delete/" + deleteId, location=loc);
         }
     }
-
 });
 
 //triggered when modal is about to be shown
@@ -333,7 +343,6 @@ $(function() {
                 } else {
                     $("#parent").addClass('hidden')
                 }
-                // console.log(data);
                 data.files.forEach(function(entry) {
                     if(entry.type === "dir") {
                         var type = "<span class=\"glyphicon glyphicon-folder-close\"></span>";
@@ -542,6 +551,7 @@ $(function() {
             $.get(e.relatedTarget.href).done(function(content) {
                 $modalBody.html(content);
                 preFilters.remove(useCache);
+                $("#back").remove();
             });
         })
         .on("hidden.bs.modal", function() {
@@ -622,8 +632,12 @@ $(function() {
            "btnfullsync",
             "GeneralDeleteModal",
             $(this).data('value'),
-            function(value){
-                path = getPath() + "/ajax/fullsync"
+            function(userid) {
+                if (userid) {
+                    path = getPath() + "/ajax/fullsync/" + userid
+                } else {
+                    path = getPath() + "/ajax/fullsync"
+                }
                 $.ajax({
                     method:"post",
                     url: path,

cps/static/js/password.js

@@ -24,7 +24,7 @@ $(document).ready(function() {
           },
 
         }, function () {
-        if ($('#password').data("verify")) {
+        if ($('#password').data("verify") === "True") {
             // Initialized and ready to go
             var options = {};
             options.common = {
@@ -38,22 +38,20 @@ $(document).ready(function() {
                 showVerdicts: false,
             }
             options.rules= {
-                specialCharClass: "(?=.*?[^A-Za-z\\s0-9])",
+                specialCharClass: "(?=.*?[^\\p{Letter}\\s0-9])",
                 activated: {
                     wordNotEmail: false,
                     wordMinLength: $('#password').data("min"),
-                    // wordMaxLength: false,
-                    // wordInvalidChar: true,
                     wordSimilarToUsername: false,
                     wordSequences: false,
                     wordTwoCharacterClasses: false,
                     wordRepetitions: false,
                     wordLowercase: $('#password').data("lower") === "True" ? true : false,
                     wordUppercase: $('#password').data("upper") === "True" ? true : false,
+                    word: $('#password').data("word") === "True" ? true : false,
                     wordOneNumber: $('#password').data("number") === "True" ? true : false,
                     wordThreeNumbers: false,
                     wordOneSpecialChar: $('#password').data("special") === "True" ? true : false,
-                    // wordTwoSpecialChar: true,
                     wordUpperLowerCombo: false,
                     wordLetterNumberCombo: false,
                     wordLetterNumberCharCombo: false

cps/tasks/convert.py

@@ -19,8 +19,10 @@
 import os
 import re
 from glob import glob
-from shutil import copyfile
+from shutil import copyfile, copyfileobj
 from markupsafe import escape
+from time import time
+from uuid import uuid4
 
 from sqlalchemy.exc import SQLAlchemyError
 from flask_babel import lazy_gettext as N_
@@ -32,13 +34,15 @@ from cps.subproc_wrapper import process_open
 from flask_babel import gettext as _
 from cps.kobo_sync_status import remove_synced_book
 from cps.ub import init_db_thread
+from cps.file_helper import get_temp_dir
 
 from cps.tasks.mail import TaskEmail
-from cps import gdriveutils
-
+from cps import gdriveutils, helper
+from cps.constants import SUPPORTED_CALIBRE_BINARIES
 
 log = logger.create()
 
+current_milli_time = lambda: int(round(time() * 1000))
 
 class TaskConvert(CalibreTask):
     def __init__(self, file_path, book_id, task_message, settings, ereader_mail, user=None):
@@ -61,24 +65,33 @@ class TaskConvert(CalibreTask):
             data = worker_db.get_book_format(self.book_id, self.settings['old_book_format'])
             df = gdriveutils.getFileFromEbooksFolder(cur_book.path,
                                                      data.name + "." + self.settings['old_book_format'].lower())
+            df_cover = gdriveutils.getFileFromEbooksFolder(cur_book.path, "cover.jpg")
             if df:
-                datafile = os.path.join(config.config_calibre_dir,
+                datafile = os.path.join(config.get_book_path(),
                                         cur_book.path,
                                         data.name + "." + self.settings['old_book_format'].lower())
-                if not os.path.exists(os.path.join(config.config_calibre_dir, cur_book.path)):
-                    os.makedirs(os.path.join(config.config_calibre_dir, cur_book.path))
+                if df_cover:
+                    datafile_cover = os.path.join(config.get_book_path(),
+                                            cur_book.path, "cover.jpg")
+                if not os.path.exists(os.path.join(config.get_book_path(), cur_book.path)):
+                    os.makedirs(os.path.join(config.get_book_path(), cur_book.path))
                 df.GetContentFile(datafile)
+                if df_cover:
+                    df_cover.GetContentFile(datafile_cover)
                 worker_db.session.close()
             else:
+                # ToDo Include cover in error handling
                 error_message = _("%(format)s not found on Google Drive: %(fn)s",
                                   format=self.settings['old_book_format'],
                                   fn=data.name + "." + self.settings['old_book_format'].lower())
                 worker_db.session.close()
-                return error_message
+                return self._handleError(self, error_message)
 
         filename = self._convert_ebook_format()
         if config.config_use_google_drive:
             os.remove(self.file_path + '.' + self.settings['old_book_format'].lower())
+            if df_cover:
+                os.remove(os.path.join(config.config_calibre_dir, cur_book.path, "cover.jpg"))
 
         if filename:
             if config.config_use_google_drive:
@@ -97,6 +110,7 @@ class TaskConvert(CalibreTask):
                                                            self.ereader_mail,
                                                            EmailText,
                                                            self.settings['body'],
+                                                           id=self.book_id,
                                                            internal=True)
                                       )
                 except Exception as ex:
@@ -112,7 +126,7 @@ class TaskConvert(CalibreTask):
 
         # check to see if destination format already exists - or if book is in database
         # if it does - mark the conversion task as complete and return a success
-        # this will allow send to E-Reader workflow to continue to work
+        # this will allow to send to E-Reader workflow to continue to work
         if os.path.isfile(file_path + format_new_ext) or\
                 local_db.get_book_format(self.book_id, self.settings['new_book_format']):
             log.info("Book id %d already converted to %s", book_id, format_new_ext)
@@ -152,7 +166,8 @@ class TaskConvert(CalibreTask):
             if not os.path.exists(config.config_converterpath):
                 self._handleError(N_("Calibre ebook-convert %(tool)s not found", tool=config.config_converterpath))
                 return
-            check, error_message = self._convert_calibre(file_path, format_old_ext, format_new_ext)
+            has_cover = local_db.get_book(book_id).has_cover
+            check, error_message = self._convert_calibre(file_path, format_old_ext, format_new_ext, has_cover)
 
         if check == 0:
             cur_book = local_db.get_book(book_id)
@@ -194,8 +209,15 @@ class TaskConvert(CalibreTask):
         return
 
     def _convert_kepubify(self, file_path, format_old_ext, format_new_ext):
+        if config.config_embed_metadata and config.config_binariesdir:
+            tmp_dir, temp_file_name = helper.do_calibre_export(self.book_id, format_old_ext[1:])
+            filename = os.path.join(tmp_dir, temp_file_name + format_old_ext)
+            temp_file_path = tmp_dir
+        else:
+            filename = file_path + format_old_ext
+            temp_file_path = os.path.dirname(file_path)
         quotes = [1, 3]
-        command = [config.config_kepubifypath, (file_path + format_old_ext), '-o', os.path.dirname(file_path)]
+        command = [config.config_kepubifypath, filename, '-o', temp_file_path, '-i']
         try:
             p = process_open(command, quotes)
         except OSError as e:
@@ -209,13 +231,12 @@ class TaskConvert(CalibreTask):
             if p.poll() is not None:
                 break
 
-        # ToD Handle
         # process returncode
         check = p.returncode
 
         # move file
         if check == 0:
-            converted_file = glob(os.path.join(os.path.dirname(file_path), "*.kepub.epub"))
+            converted_file = glob(os.path.splitext(filename)[0] + "*.kepub.epub")
             if len(converted_file) == 1:
                 copyfile(converted_file[0], (file_path + format_new_ext))
                 os.unlink(converted_file[0])
@@ -224,16 +245,35 @@ class TaskConvert(CalibreTask):
                             folder=os.path.dirname(file_path))
         return check, None
 
-    def _convert_calibre(self, file_path, format_old_ext, format_new_ext):
+    def _convert_calibre(self, file_path, format_old_ext, format_new_ext, has_cover):
         try:
-            # Linux py2.7 encode as list without quotes no empty element for parameters
-            # linux py3.x no encode and as list without quotes no empty element for parameters
-            # windows py2.7 encode as string with quotes empty element for parameters is okay
-            # windows py 3.x no encode and as string with quotes empty element for parameters is okay
-            # separate handling for windows and linux
-            quotes = [1, 2]
+            # path_tmp_opf = self._embed_metadata()
+            if config.config_embed_metadata:
+                quotes = [3, 5]
+                tmp_dir = get_temp_dir()
+                calibredb_binarypath = os.path.join(config.config_binariesdir, SUPPORTED_CALIBRE_BINARIES["calibredb"])
+                my_env = os.environ.copy()
+                if config.config_calibre_split:
+                    my_env['CALIBRE_OVERRIDE_DATABASE_PATH'] = os.path.join(config.config_calibre_dir, "metadata.db")
+                    library_path = config.config_calibre_split_dir
+                else:
+                    library_path = config.config_calibre_dir
+
+                opf_command = [calibredb_binarypath, 'show_metadata', '--as-opf', str(self.book_id),
+                               '--with-library', library_path]
+                p = process_open(opf_command, quotes, my_env)
+                p.wait()
+                path_tmp_opf = os.path.join(tmp_dir, "metadata_" + str(uuid4()) + ".opf")
+                with open(path_tmp_opf, 'w') as fd:
+                    copyfileobj(p.stdout, fd)
+
+            quotes = [1, 2, 4, 6]
             command = [config.config_converterpath, (file_path + format_old_ext),
                        (file_path + format_new_ext)]
+            if config.config_embed_metadata:
+                command.extend(['--from-opf', path_tmp_opf])
+            if has_cover:
+                command.extend(['--cover', os.path.join(os.path.dirname(file_path), 'cover.jpg')])
             quotes_index = 3
             if config.config_calibre:
                 parameters = config.config_calibre.split(" ")
@@ -276,9 +316,9 @@ class TaskConvert(CalibreTask):
 
     def __str__(self):
         if self.ereader_mail:
-            return "Convert {} {}".format(self.book_id, self.ereader_mail)
+            return "Convert Book {} and mail it to {}".format(self.book_id, self.ereader_mail)
         else:
-            return "Convert {}".format(self.book_id)
+            return "Convert Book {}".format(self.book_id)
 
     @property
     def is_cancellable(self):

cps/tasks/mail.py

@@ -18,6 +18,7 @@
 
 import os
 import smtplib
+import ssl
 import threading
 import socket
 import mimetypes
@@ -27,12 +28,11 @@ from email.message import EmailMessage
 from email.utils import formatdate, parseaddr
 from email.generator import Generator
 from flask_babel import lazy_gettext as N_
-from email.utils import formatdate
 
 from cps.services.worker import CalibreTask
 from cps.services import gmail
+from cps.embed_helper import do_calibre_export
 from cps import logger, config
-
 from cps import gdriveutils
 import uuid
 
@@ -109,7 +109,7 @@ class EmailSSL(EmailBase, smtplib.SMTP_SSL):
 
 
 class TaskEmail(CalibreTask):
-    def __init__(self, subject, filepath, attachment, settings, recipient, task_message, text, internal=False):
+    def __init__(self, subject, filepath, attachment, settings, recipient, task_message, text, id=0, internal=False):
         super(TaskEmail, self).__init__(task_message)
         self.subject = subject
         self.attachment = attachment
@@ -118,6 +118,7 @@ class TaskEmail(CalibreTask):
         self.recipient = recipient
         self.text = text
         self.asyncSMTP = None
+        self.book_id = id
         self.results = dict()
 
     # from calibre code:
@@ -140,7 +141,7 @@ class TaskEmail(CalibreTask):
         message['To'] = self.recipient
         message['Subject'] = self.subject
         message['Date'] = formatdate(localtime=True)
-        message['Message-Id'] = "{}@{}".format(uuid.uuid4(), self.get_msgid_domain()) # f"<{uuid.uuid4()}@{get_msgid_domain(from_)}>" # make_msgid('calibre-web')
+        message['Message-Id'] = "{}@{}".format(uuid.uuid4(), self.get_msgid_domain())
         message.set_content(self.text.encode('UTF-8'), "text", "plain")
         if self.attachment:
             data = self._get_attachment(self.filepath, self.attachment)
@@ -160,6 +161,8 @@ class TaskEmail(CalibreTask):
         try:
             # create MIME message
             msg = self.prepare_message()
+            if not msg:
+                return
             if self.settings['mail_server_type'] == 0:
                 self.send_standard_email(msg)
             else:
@@ -192,8 +195,9 @@ class TaskEmail(CalibreTask):
         # on python3 debugoutput is caught with overwritten _print_debug function
         log.debug("Start sending e-mail")
         if use_ssl == 2:
+            context = ssl.create_default_context()
             self.asyncSMTP = EmailSSL(self.settings["mail_server"], self.settings["mail_port"],
-                                       timeout=timeout)
+                                       timeout=timeout, context=context)
         else:
             self.asyncSMTP = Email(self.settings["mail_server"], self.settings["mail_port"], timeout=timeout)
 
@@ -201,7 +205,8 @@ class TaskEmail(CalibreTask):
         if logger.is_debug_enabled():
             self.asyncSMTP.set_debuglevel(1)
         if use_ssl == 1:
-            self.asyncSMTP.starttls()
+            context = ssl.create_default_context()
+            self.asyncSMTP.starttls(context=context)
         if self.settings["mail_password_e"]:
             self.asyncSMTP.login(str(self.settings["mail_login"]), str(self.settings["mail_password_e"]))
 
@@ -233,10 +238,10 @@ class TaskEmail(CalibreTask):
             self.asyncSMTP = None
             self._progress = x
 
-    @classmethod
-    def _get_attachment(cls, book_path, filename):
+    def _get_attachment(self, book_path, filename):
         """Get file as MIMEBase message"""
-        calibre_path = config.config_calibre_dir
+        calibre_path = config.get_book_path()
+        extension = os.path.splitext(filename)[1][1:]
         if config.config_use_google_drive:
             df = gdriveutils.getFileFromEbooksFolder(book_path, filename)
             if df:
@@ -246,15 +251,22 @@ class TaskEmail(CalibreTask):
                 df.GetContentFile(datafile)
             else:
                 return None
-            file_ = open(datafile, 'rb')
-            data = file_.read()
-            file_.close()
+            if config.config_binariesdir and config.config_embed_metadata:
+                data_path, data_file = do_calibre_export(self.book_id, extension)
+                datafile = os.path.join(data_path, data_file + "." + extension)
+            with open(datafile, 'rb') as file_:
+                data = file_.read()
             os.remove(datafile)
         else:
+            datafile = os.path.join(calibre_path, book_path, filename)
             try:
-                file_ = open(os.path.join(calibre_path, book_path, filename), 'rb')
-                data = file_.read()
-                file_.close()
+                if config.config_binariesdir and config.config_embed_metadata:
+                    data_path, data_file = do_calibre_export(self.book_id, extension)
+                    datafile = os.path.join(data_path, data_file + "." + extension)
+                with open(datafile, 'rb') as file_:
+                    data = file_.read()
+                if config.config_binariesdir and config.config_embed_metadata:
+                    os.remove(datafile)
             except IOError as e:
                 log.error_or_exception(e, stacklevel=3)
                 log.error('The requested file could not be read. Maybe wrong permissions?')

cps/tasks/metadata_backup.py

@@ -17,26 +17,13 @@
 #   along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 import os
-from urllib.request import urlopen
 from lxml import etree
 
-
 from cps import config, db, gdriveutils, logger
 from cps.services.worker import CalibreTask
 from flask_babel import lazy_gettext as N_
 
-OPF_NAMESPACE = "http://www.idpf.org/2007/opf"
-PURL_NAMESPACE = "http://purl.org/dc/elements/1.1/"
-
-OPF = "{%s}" % OPF_NAMESPACE
-PURL = "{%s}" % PURL_NAMESPACE
-
-etree.register_namespace("opf", OPF_NAMESPACE)
-etree.register_namespace("dc", PURL_NAMESPACE)
-
-OPF_NS = {None: OPF_NAMESPACE}  # the default namespace (no prefix)
-NSMAP = {'dc': PURL_NAMESPACE, 'opf': OPF_NAMESPACE}
-
+from ..epub_helper import create_new_metadata_backup
 
 class TaskBackupMetadata(CalibreTask):
 
@@ -101,7 +88,8 @@ class TaskBackupMetadata(CalibreTask):
             self.calibre_db.session.close()
 
     def open_metadata(self, book, custom_columns):
-        package = self.create_new_metadata_backup(book, custom_columns)
+        # package = self.create_new_metadata_backup(book, custom_columns)
+        package = create_new_metadata_backup(book, custom_columns, self.export_language, self.translated_title)
         if config.config_use_google_drive:
             if not gdriveutils.is_gdrive_ready():
                 raise Exception('Google Drive is configured but not ready')
@@ -114,7 +102,7 @@ class TaskBackupMetadata(CalibreTask):
                                                  True)
         else:
             # ToDo: Handle book folder not found or not readable
-            book_metadata_filepath = os.path.join(config.config_calibre_dir, book.path, 'metadata.opf')
+            book_metadata_filepath = os.path.join(config.get_book_path(), book.path, 'metadata.opf')
             # prepare finalize everything and output
             doc = etree.ElementTree(package)
             try:
@@ -123,93 +111,6 @@ class TaskBackupMetadata(CalibreTask):
             except Exception as ex:
                 raise Exception('Writing Metadata failed with error: {} '.format(ex))
 
-    def create_new_metadata_backup(self, book,  custom_columns):
-        # generate root package element
-        package = etree.Element(OPF + "package", nsmap=OPF_NS)
-        package.set("unique-identifier", "uuid_id")
-        package.set("version", "2.0")
-
-        # generate metadata element and all sub elements of it
-        metadata = etree.SubElement(package, "metadata", nsmap=NSMAP)
-        identifier = etree.SubElement(metadata, PURL + "identifier", id="calibre_id", nsmap=NSMAP)
-        identifier.set(OPF + "scheme", "calibre")
-        identifier.text = str(book.id)
-        identifier2 = etree.SubElement(metadata, PURL + "identifier", id="uuid_id", nsmap=NSMAP)
-        identifier2.set(OPF + "scheme", "uuid")
-        identifier2.text = book.uuid
-        title = etree.SubElement(metadata, PURL + "title", nsmap=NSMAP)
-        title.text = book.title
-        for author in book.authors:
-            creator = etree.SubElement(metadata, PURL + "creator", nsmap=NSMAP)
-            creator.text = str(author.name)
-            creator.set(OPF + "file-as", book.author_sort)     # ToDo Check
-            creator.set(OPF + "role", "aut")
-        contributor = etree.SubElement(metadata, PURL + "contributor", nsmap=NSMAP)
-        contributor.text = "calibre (5.7.2) [https://calibre-ebook.com]"
-        contributor.set(OPF + "file-as", "calibre")     # ToDo Check
-        contributor.set(OPF + "role", "bkp")
-
-        date = etree.SubElement(metadata, PURL + "date", nsmap=NSMAP)
-        date.text = '{d.year:04}-{d.month:02}-{d.day:02}T{d.hour:02}:{d.minute:02}:{d.second:02}'.format(d=book.pubdate)
-        if book.comments and book.comments[0].text:
-            for b in book.comments:
-                description = etree.SubElement(metadata, PURL + "description", nsmap=NSMAP)
-                description.text = b.text
-        for b in book.publishers:
-            publisher = etree.SubElement(metadata, PURL + "publisher", nsmap=NSMAP)
-            publisher.text = str(b.name)
-        if not book.languages:
-            language = etree.SubElement(metadata, PURL + "language", nsmap=NSMAP)
-            language.text = self.export_language
-        else:
-            for b in book.languages:
-                language = etree.SubElement(metadata, PURL + "language", nsmap=NSMAP)
-                language.text = str(b.lang_code)
-        for b in book.tags:
-            subject = etree.SubElement(metadata, PURL + "subject", nsmap=NSMAP)
-            subject.text = str(b.name)
-        etree.SubElement(metadata, "meta", name="calibre:author_link_map",
-                         content="{" + ", ".join(['"' + str(a.name) + '": ""' for a in book.authors]) + "}",
-                         nsmap=NSMAP)
-        for b in book.series:
-            etree.SubElement(metadata, "meta", name="calibre:series",
-                             content=str(str(b.name)),
-                             nsmap=NSMAP)
-        if book.series:
-            etree.SubElement(metadata, "meta", name="calibre:series_index",
-                             content=str(book.series_index),
-                             nsmap=NSMAP)
-        if len(book.ratings) and book.ratings[0].rating > 0:
-            etree.SubElement(metadata, "meta", name="calibre:rating",
-                             content=str(book.ratings[0].rating),
-                             nsmap=NSMAP)
-        etree.SubElement(metadata, "meta", name="calibre:timestamp",
-                         content='{d.year:04}-{d.month:02}-{d.day:02}T{d.hour:02}:{d.minute:02}:{d.second:02}'.format(
-                             d=book.timestamp),
-                         nsmap=NSMAP)
-        etree.SubElement(metadata, "meta", name="calibre:title_sort",
-                         content=book.sort,
-                         nsmap=NSMAP)
-        sequence = 0
-        for cc in custom_columns:
-            value = None
-            extra = None
-            cc_entry = getattr(book, "custom_column_" + str(cc.id))
-            if cc_entry.__len__():
-                value = [c.value for c in cc_entry] if cc.is_multiple else cc_entry[0].value
-                extra = cc_entry[0].extra if hasattr(cc_entry[0], "extra") else None
-            etree.SubElement(metadata, "meta", name="calibre:user_metadata:#{}".format(cc.label),
-                             content=cc.to_json(value, extra, sequence),
-                             nsmap=NSMAP)
-            sequence += 1
-
-        # generate guide element and all sub elements of it
-        # Title is translated from default export language
-        guide = etree.SubElement(package, "guide")
-        etree.SubElement(guide, "reference", type="cover", title=self.translated_title, href="cover.jpg")
-
-        return package
-
     @property
     def name(self):
         return "Metadata backup"

cps/tasks/tempFolder.py

@@ -0,0 +1,47 @@
+# -*- coding: utf-8 -*-
+
+#   This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#     Copyright (C) 2023 OzzieIsaacs
+#
+#   This program is free software: you can redistribute it and/or modify
+#   it under the terms of the GNU General Public License as published by
+#   the Free Software Foundation, either version 3 of the License, or
+#   (at your option) any later version.
+#
+#   This program is distributed in the hope that it will be useful,
+#   but WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#   GNU General Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License
+#   along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from urllib.request import urlopen
+
+from flask_babel import lazy_gettext as N_
+
+from cps import logger, file_helper
+from cps.services.worker import CalibreTask
+
+
+class TaskDeleteTempFolder(CalibreTask):
+    def __init__(self, task_message=N_('Delete temp folder contents')):
+        super(TaskDeleteTempFolder, self).__init__(task_message)
+        self.log = logger.create()
+
+    def run(self, worker_thread):
+        try:
+            file_helper.del_temp_dir()
+        except FileNotFoundError:
+            pass
+        except (PermissionError, OSError) as e:
+            self.log.error("Error deleting temp folder: {}".format(e))
+        self._handleSuccess()
+
+    @property
+    def name(self):
+        return "Delete Temp Folder"
+
+    @property
+    def is_cancellable(self):
+        return False

cps/tasks/thumbnail.py

@@ -138,7 +138,7 @@ class TaskGenerateCoverThumbnails(CalibreTask):
 
         # Replace outdated or missing thumbnails
         for thumbnail in book_cover_thumbnails:
-            if book.last_modified > thumbnail.generated_at:
+            if book.last_modified.replace(tzinfo=None) > thumbnail.generated_at:
                 generated += 1
                 self.update_book_cover_thumbnail(book, thumbnail)
 
@@ -209,7 +209,7 @@ class TaskGenerateCoverThumbnails(CalibreTask):
                     if stream is not None:
                         stream.close()
             else:
-                book_cover_filepath = os.path.join(config.config_calibre_dir, book.path, 'cover.jpg')
+                book_cover_filepath = os.path.join(config.get_book_path(), book.path, 'cover.jpg')
                 if not os.path.isfile(book_cover_filepath):
                     raise Exception('Book cover file not found')
 
@@ -404,7 +404,7 @@ class TaskGenerateSeriesThumbnails(CalibreTask):
                         if stream is not None:
                             stream.close()
 
-                book_cover_filepath = os.path.join(config.config_calibre_dir, book.path, 'cover.jpg')
+                book_cover_filepath = os.path.join(config.get_book_path(), book.path, 'cover.jpg')
                 if not os.path.isfile(book_cover_filepath):
                     raise Exception('Book cover file not found')
 

cps/tasks_status.py

@@ -43,9 +43,7 @@ def get_email_status_json():
 @login_required
 def get_tasks_status():
     # if current user admin, show all email, otherwise only own emails
-    tasks = WorkerThread.get_instance().tasks
-    answer = render_task_status(tasks)
-    return render_title_template('tasks.html', entries=answer, title=_("Tasks"), page="tasks")
+    return render_title_template('tasks.html', title=_("Tasks"), page="tasks")
 
 
 # helper function to apply localize status information in tasklist entries

cps/templates/author.html

@@ -8,8 +8,8 @@
   <img title="{{author.name}}" src="{{author.image_url}}" alt="{{author.name}}" class="author-photo pull-left">
   {% endif %}
 
-  {%if author.about is not none %}
-  <p>{{author.about}}</p>
+  {%if author.safe_about is not none %}
+  <p>{{author.safe_about|safe}}</p>
   {% endif %}
 
   - {{_("via")}} <a href="{{author.link}}" class="author-link" target="_blank" rel="noopener">Goodreads</a>
@@ -32,7 +32,7 @@
     </div>
   <div class="row display-flex">
     {% for entry in entries %}
-    <div id="books" class="col-sm-3 col-lg-2 col-xs-6 book">
+    <div id="books" class="col-sm-3 col-lg-2 col-xs-6 book session">
       <div class="cover">
         <a href="{{ url_for('web.show_book', book_id=entry.Books.id) }}" {% if simple==false %}data-toggle="modal" data-target="#bookDetailsModal" data-remote="false"{% endif %}>
             <span class="img" title="{{entry.Books.title}}">
@@ -99,7 +99,7 @@
   <h3>{{_("More by")}} {{ author.name.replace('|',',') }}</h3>
   <div class="row">
     {% for entry in other_books %}
-    <div class="col-sm-3 col-lg-2 col-xs-6 book">
+    <div class="col-sm-3 col-lg-2 col-xs-6 book session">
       <div class="cover">
         <a href="https://www.goodreads.com/book/show/{{ entry.gid['#text'] }}" target="_blank" rel="noopener">
           <img title="{{entry.title}}" src="{{ entry.image_url }}" />

cps/templates/book_edit.html

@@ -61,7 +61,7 @@
     </div>
     <div id="author_div" class="form-group">
       <label for="bookAuthor">{{_('Author')}}</label>
-      <input type="text" class="form-control typeahead" name="author_name" id="bookAuthor" value="{{' & '.join(authors)}}" autocomplete="off">
+      <input type="text" class="form-control typeahead" autocomplete="off" name="author_name" id="bookAuthor" value="{{' & '.join(authors)}}">
     </div>
 
     <div class="form-group">
@@ -85,11 +85,11 @@
 
     <div class="form-group">
       <label for="tags">{{_('Tags')}}</label>
-      <input type="text" class="form-control typeahead" name="tags" id="tags" value="{% for tag in book.tags %}{{tag.name.strip()}}{% if not loop.last %}, {% endif %}{% endfor %}">
+      <input type="text" class="form-control typeahead" autocomplete="off"  name="tags" id="tags" value="{% for tag in book.tags %}{{tag.name.strip()}}{% if not loop.last %}, {% endif %}{% endfor %}">
     </div>
     <div class="form-group">
       <label for="series">{{_('Series')}}</label>
-      <input type="text" class="form-control typeahead" name="series" id="series" value="{% if book.series %}{{book.series[0].name}}{% endif %}">
+      <input type="text" class="form-control typeahead" autocomplete="off" name="series" id="series" value="{% if book.series %}{{book.series[0].name}}{% endif %}">
     </div>
     <div class="form-group">
       <label for="series_index">{{_('Series ID')}}</label>
@@ -120,11 +120,11 @@
     </div>
     <div class="form-group">
       <label for="publisher">{{_('Publisher')}}</label>
-      <input type="text" class="form-control typeahead" name="publisher" id="publisher" value="{% if book.publishers|length > 0 %}{{book.publishers[0].name}}{% endif %}">
+      <input type="text" class="form-control typeahead" autocomplete="off" name="publisher" id="publisher" value="{% if book.publishers|length > 0 %}{{book.publishers[0].name}}{% endif %}">
     </div>
     <div class="form-group">
       <label for="languages">{{_('Language')}}</label>
-      <input type="text" class="form-control typeahead" name="languages" id="languages" value="{% for language in book.languages %}{{language.language_name.strip()}}{% if not loop.last %}, {% endif %}{% endfor %}">
+      <input type="text" class="form-control typeahead" autocomplete="off" name="languages" id="languages" value="{% for language in book.languages %}{{language.language_name.strip()}}{% if not loop.last %}, {% endif %}{% endfor %}">
     </div>
       {% if cc|length > 0 %}
         {% for c in cc %}

cps/templates/config_db.html

@@ -16,6 +16,18 @@
           <button type="button" data-toggle="modal" id="calibre_modal_path" data-link="config_calibre_dir" data-filefilter="metadata.db" data-target="#fileModal" id="library_path" class="btn btn-default"><span class="glyphicon glyphicon-folder-open"></span></button>
         </span>
       </div>
+      <div class="form-group required">
+        <input type="checkbox" id="config_calibre_split" name="config_calibre_split" data-control="split_settings" data-t ="{{ config.config_calibre_split_dir  }}" {% if config.config_calibre_split %}checked{% endif %} >
+        <label for="config_calibre_split">{{_('Separate Book Files from Library')}}</label>
+      </div>
+      <div data-related="split_settings">
+       <div class="form-group required input-group">
+        <input type="text" class="form-control" id="config_calibre_split_dir" name="config_calibre_split_dir" value="{% if config.config_calibre_split_dir != None %}{{ config.config_calibre_split_dir }}{% endif %}" autocomplete="off">
+        <span class="input-group-btn">
+          <button type="button" data-toggle="modal" id="calibre_modal_split_path" data-link="config_calibre_split_dir" data-filefilter="" data-target="#fileModal" id="book_path" class="btn btn-default"><span class="glyphicon glyphicon-folder-open"></span></button>
+        </span>
+      </div>
+      </div>
     {% if feature_support['gdrive'] %}
       <div class="form-group required">
         <input type="checkbox" id="config_use_google_drive" name="config_use_google_drive" data-control="gdrive_settings" {% if config.config_use_google_drive %}checked{% endif %} >

cps/templates/config_edit.html

@@ -9,7 +9,7 @@
   <h2>{{title}}</h2>
 <form role="form" method="POST" autocomplete="off">
 <input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
-<div class="panel-group col-md-10 col-lg-8">
+<div class="panel-group col-md-11 col-lg-8">
   <div class="panel panel-default">
     <div class="panel-heading">
       <h4 class="panel-title">
@@ -103,9 +103,13 @@
       <input type="checkbox" id="config_unicode_filename" name="config_unicode_filename" {% if config.config_unicode_filename %}checked{% endif %}>
       <label for="config_unicode_filename">{{_('Convert non-English characters in title and author while saving to disk')}}</label>
     </div>
+    <div class="form-group">
+      <input type="checkbox" id="config_embed_metadata" name="config_embed_metadata" {% if config.config_embed_metadata %}checked{% endif %}>
+      <label for="config_embed_metadata">{{_('Embed Metadata to Ebook File on Download/Conversion/e-mail (needs Calibre/Kepubify binaries)')}}</label>
+    </div>
     <div class="form-group">
         <input type="checkbox" id="config_uploading" data-control="upload_settings" name="config_uploading" {% if config.config_uploading %}checked{% endif %}>
-        <label for="config_uploading">{{_('Enable Uploads')}} {{_('(Please ensure users having also upload rights)')}}</label>
+        <label for="config_uploading">{{_('Enable Uploads')}} {{_('(Please ensure that users also have upload permissions)')}}</label>
     </div>
     <div data-related="upload_settings">
       <div class="form-group">
@@ -151,17 +155,12 @@
     <div class="form-group">
       <input type="checkbox" id="config_use_goodreads" name="config_use_goodreads" data-control="goodreads-settings" {% if config.config_use_goodreads %}checked{% endif %}>
       <label for="config_use_goodreads">{{_('Use Goodreads')}}</label>
-      <a href="https://www.goodreads.com/api/keys" target="_blank" style="margin-left: 5px">{{_('Create an API Key')}}</a>
     </div>
     <div data-related="goodreads-settings">
       <div class="form-group">
         <label for="config_goodreads_api_key">{{_('Goodreads API Key')}}</label>
         <input type="text" class="form-control" id="config_goodreads_api_key" name="config_goodreads_api_key" value="{% if config.config_goodreads_api_key != None %}{{ config.config_goodreads_api_key }}{% endif %}" autocomplete="off">
       </div>
-      <div class="form-group">
-        <label for="config_goodreads_api_secret_e">{{_('Goodreads API Secret')}}</label>
-        <input type="password" class="form-control" id="config_goodreads_api_secret_e" name="config_goodreads_api_secret_e" value="{% if config.config_goodreads_api_secret_e != None %}{{ config.config_goodreads_api_secret_e }}{% endif %}" autocomplete="off">
-      </div>
     </div>
     {% endif %}
     <div class="form-group">
@@ -323,12 +322,12 @@
     </div>
     <div id="collapsefive" class="panel-collapse collapse">
       <div class="panel-body">
-           <label for="config_converterpath">{{_('Path to Calibre E-Book Converter')}}</label>
+           <label for="config_binariesdir">{{_('Path to Calibre Binaries')}}</label>
            <div class="form-group input-group">
-            <input type="text" class="form-control" id="config_converterpath" name="config_converterpath" value="{% if config.config_converterpath != None %}{{ config.config_converterpath }}{% endif %}" autocomplete="off">
-            <span class="input-group-btn">
-              <button type="button" data-toggle="modal" id="converter_modal_path" data-link="config_converterpath" data-target="#fileModal" class="btn btn-default"><span class="glyphicon glyphicon-folder-open"></span></button>
-            </span>
+           <input type="text" class="form-control" id="config_binariesdir" name="config_binariesdir" value="{% if config.config_binariesdir != None %}{{ config.config_binariesdir }}{% endif %}" autocomplete="off">
+           <span class="input-group-btn">
+             <button type="button" data-toggle="modal" id="binaries_modal_path" data-link="config_binariesdir" data-folderonly="true" data-target="#fileModal" class="btn btn-default"><span class="glyphicon glyphicon-folder-open"></span></button>
+           </span>
            </div>
            <div class="form-group">
               <label for="config_calibre">{{_('Calibre E-Book Converter Settings')}}</label>
@@ -358,7 +357,7 @@
       <h4 class="panel-title">
         <a class="accordion-toggle" data-toggle="collapse" href="#collapsesix">
           <span class="glyphicon glyphicon-plus"></span>
-           {{_('Securitiy Settings')}}
+           {{_('Security Settings')}}
         </a>
       </h4>
     </div>
@@ -368,6 +367,16 @@
               <input type="checkbox" id="config_ratelimiter" name="config_ratelimiter" {% if config.config_ratelimiter %}checked{% endif %}>
               <label for="config_ratelimiter">{{_('Limit failed login attempts')}}</label>
             </div>
+            <div data-related="ratelimiter_settings">
+               <div class="form-group" style="margin-left:10px;">
+                  <label for="config_calibre">{{_('Configure Backend for Limiter')}}</label>
+                  <input type="text" class="form-control" id="config_limiter_uri" name="config_limiter_uri" value="{% if config.config_limiter_uri != None %}{{ config.config_limiter_uri }}{% endif %}" autocomplete="off">
+               </div>
+               <div class="form-group" style="margin-left:10px;">
+                  <label for="config_calibre">{{_('Options for Limiter')}}</label>
+                  <input type="text" class="form-control" id="config_limiter_options" name="config_limiter_options" value="{% if config.config_limiter_options != None %}{{ config.config_limiter_options }}{% endif %}" autocomplete="off">
+               </div>
+            </div>
             <div class="form-group">
               <label for="config_session">{{_('Session protection')}}</label>
                 <select name="config_session" id="config_session" class="form-control">
@@ -396,6 +405,10 @@
                 <input type="checkbox" id="config_password_upper" name="config_password_upper"  {% if config.config_password_upper %}checked{% endif %}>
                 <label for="config_password_upper">{{_('Enforce uppercase characters')}}</label>
               </div>
+              <div class="form-group" style="margin-left:10px;">
+                <input type="checkbox" id="config_password_character" name="config_password_character"  {% if config.config_password_character %}checked{% endif %}>
+                <label for="config_password_lower">{{_('Enforce characters (needed For Chinese/Japanese/Korean Characters)')}}</label>
+              </div>
               <div class="form-group" style="margin-left:10px;">
                 <input type="checkbox" id="config_password_special" name="config_password_special"  {% if config.config_password_special %}checked{% endif %}>
                 <label for="config_password_special">{{_('Enforce special characters')}}</label>

cps/templates/detail.html

@@ -43,30 +43,30 @@
                                 {% endif %}
                             </div>
                         {% endif %}
-                    {% endif %}
-                    {% if current_user.kindle_mail and entry.email_share_list %}
-                        <input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
-                        {% if entry.email_share_list.__len__() == 1 %}
-                            <div class="btn-group" role="group">
-                                <button id="sendbtn" class="btn btn-primary sendbtn-form" data-href="{{url_for('web.send_to_ereader', book_id=entry.id, book_format=entry.email_share_list[0]['format'], convert=entry.email_share_list[0]['convert'])}}">
-                                    <span class="glyphicon glyphicon-send"></span> {{entry.email_share_list[0]['text']}}
-                                </button>
-                            </div>
-                        {% else %}
-                            <div class="btn-group" role="group">
-                                <button id="sendbtn2" type="button" class="btn btn-primary dropdown-toggle"
-                                        data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
-                                    <span class="glyphicon glyphicon-send"></span>{{ _('Send to eReader') }}
-                                    <span class="caret"></span>
-                                </button>
-                                <ul class="dropdown-menu" aria-labelledby="send-to-ereader">
-                                    {% for format in entry.email_share_list %}
-                                        <li>
-                                            <a class="sendbtn-form" data-href="{{url_for('web.send_to_ereader', book_id=entry.id, book_format=format['format'], convert=format['convert'])}}">{{ format['text'] }}</a>
-                                        </li>
-                                    {% endfor %}
-                                </ul>
-                            </div>
+                        {% if current_user.kindle_mail and entry.email_share_list %}
+                            <input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
+                            {% if entry.email_share_list.__len__() == 1 %}
+                                <div class="btn-group" role="group">
+                                    <button id="sendbtn" class="btn btn-primary sendbtn-form" data-href="{{url_for('web.send_to_ereader', book_id=entry.id, book_format=entry.email_share_list[0]['format'], convert=entry.email_share_list[0]['convert'])}}">
+                                        <span class="glyphicon glyphicon-send"></span> {{entry.email_share_list[0]['text']}}
+                                    </button>
+                                </div>
+                            {% else %}
+                                <div class="btn-group" role="group">
+                                    <button id="sendbtn2" type="button" class="btn btn-primary dropdown-toggle"
+                                            data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
+                                        <span class="glyphicon glyphicon-send"></span>{{ _('Send to eReader') }}
+                                        <span class="caret"></span>
+                                    </button>
+                                    <ul class="dropdown-menu" aria-labelledby="send-to-ereader">
+                                        {% for format in entry.email_share_list %}
+                                            <li>
+                                                <a class="sendbtn-form" data-href="{{url_for('web.send_to_ereader', book_id=entry.id, book_format=format['format'], convert=format['convert'])}}">{{ format['text'] }}</a>
+                                            </li>
+                                        {% endfor %}
+                                    </ul>
+                                </div>
+                            {% endif %}
                         {% endif %}
                     {% endif %}
                     {% if entry.reader_list and current_user.role_viewer() %}
@@ -164,7 +164,7 @@
                     <p>
                         <span class="glyphicon glyphicon-link"></span>
                         {% for identifier in entry.identifiers %}
-                            <a href="{{ identifier }}" target="_blank" class="btn btn-xs btn-success"
+                            <a href="{{ identifier|escape }}" target="_blank" class="btn btn-xs btn-success"
                                role="button">{{ identifier.format_type() }}</a>
                         {% endfor %}
                     </p>
@@ -205,8 +205,8 @@
 
 
                 {% for c in cc %}
-                    <div class="real_custom_columns">
-                        {% if entry['custom_column_' ~ c.id]|length > 0 %}
+                    {% if entry['custom_column_' ~ c.id]|length > 0 %}
+                        <div class="real_custom_columns">
                             {{ c.name }}:
                             {% for column in entry['custom_column_' ~ c.id] %}
                                 {% if c.datatype == 'rating' %}
@@ -235,8 +235,9 @@
                                     {% endif %}
                                 {% endif %}
                             {% endfor %}
-                        {% endif %}
-                    </div>
+
+                        </div>
+                    {% endif %}
                 {% endfor %}
             {% endif %}
             {% if not current_user.is_anonymous %}
@@ -332,15 +333,15 @@
 
                 {% endif %}
                 {% if current_user.role_edit() %}
-                    <div class="btn-toolbar" role="toolbar">
+                    <div class="col-sm-12">
                         <div class="btn-group" role="group" aria-label="Edit/Delete book">
                             <a href="{{ url_for('edit-book.show_edit_book', book_id=entry.id) }}"
                                class="btn btn-sm btn-primary" id="edit_book" role="button"><span
                                     class="glyphicon glyphicon-edit"></span> {{ _('Edit Metadata') }}</a>
                         </div>
-                    </div>
+                        <div class="btn btn-default" data-back="{{ url_for('web.index') }}" id="back">{{_('Cancel')}}</div>
+                   </div>
                 {% endif %}
-            </div>
         </div>
     </div>
 </div>
@@ -366,4 +367,3 @@
 </script>
 
 {% endblock %}
-

cps/templates/email_edit.html

@@ -49,7 +49,7 @@
       </div>
       <div class="form-group">
         <label for="mail_password_e">{{_('SMTP Password')}}</label>
-        <input type="password" class="form-control" name="mail_password_e" id="mail_password_e" value="{{content.mail_password_e}}">
+        <input type="password" class="form-control" name="mail_password_e" id="mail_password_e" value="">
       </div>
       <div class="form-group">
         <label for="mail_from">{{_('From Email')}}</label>

cps/templates/feed.xml

@@ -1,5 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <feed xmlns="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/terms/" xmlns:dcterms="http://purl.org/dc/terms/">
+  <icon>{{ url_for('static', filename='favicon.ico') }}</icon>
   <id>urn:uuid:2853dacf-ed79-42f5-8e8a-a7bb3d1ae6a2</id>
   <updated>{{ current_time }}</updated>
   <link rel="self"
@@ -30,7 +31,7 @@
     <link rel="search"
       href="{{url_for('opds.feed_osd')}}"
       type="application/opensearchdescription+xml"/>
-  <link type="application/atom+xml" rel="search" title="{{_('Search')}}" href="{{url_for('opds.feed_cc_search')}}/{searchTerms}" />
+  <link type="application/atom+xml" rel="search" title="{{_('Search')}}" href="{{url_for('opds.feed_normal_search')}}/{searchTerms}" />
   <title>{{instance}}</title>
   <author>
     <name>{{instance}}</name>

cps/templates/grid.html

@@ -28,7 +28,7 @@
                   <div class="cover">
                       <a href="{{url_for('web.books_list', data=data, sort_param='stored', book_id=entry[0].series[0].id )}}">
                           <span class="img" title="{{entry[0].series[0].name}}">
-                              {{ image.series(entry[0].series[0], alt=entry[0].series[0].name|shortentitle) }}
+                              {{ image.book_cover(entry[0])}}
                               <span class="badge">{{entry.count}}</span>
                             </span>
                       </a>

cps/templates/http_error.html

@@ -20,7 +20,7 @@
     {% endif %}
   </head>
   <body>
-  <div class="container-fluid">
+  <div class="container-fluid" style="overflow-y: auto">
   <div class="row">
     <div class="col">
         <h1  class="text-center">{{instance}}</h1>
@@ -41,7 +41,7 @@
   {% if issue %}
     <div class="row">
     <div class="col errorlink">Please report this issue with all related information:
-        <a href="https://github.com/janeczku/calibre-web/issues/new/choose=">{{_('Create Issue')}}</a>
+        <a href="https://github.com/janeczku/calibre-web/issues/new/choose">{{_('Create Issue')}}</a>
     </div>
       </div>
     {% endif %}

cps/templates/index.html

@@ -6,7 +6,7 @@
   <h2 class="random-books">{{_('Discover (Random Books)')}}</h2>
   <div class="row display-flex">
    {% for entry in random %}
-    <div class="col-sm-3 col-lg-2 col-xs-6 book" id="books_rand">
+    <div class="col-sm-3 col-lg-2 col-xs-6 book session" id="books_rand">
       <div class="cover">
           <a href="{{ url_for('web.show_book', book_id=entry.Books.id) }}" {% if simple==false %}data-toggle="modal" data-target="#bookDetailsModal" data-remote="false"{% endif %}>
               <span class="img" title="{{ entry.Books.title }}">
@@ -89,7 +89,7 @@
   <div class="row display-flex">
     {% if entries[0] %}
     {% for entry in entries %}
-    <div class="col-sm-3 col-lg-2 col-xs-6 book" id="books">
+    <div class="col-sm-3 col-lg-2 col-xs-6 book session" id="books">
       <div class="cover">
           <a href="{{ url_for('web.show_book', book_id=entry.Books.id) }}" {% if simple==false %}data-toggle="modal" data-target="#bookDetailsModal" data-remote="false"{% endif %}>
             <span class="img" title="{{ entry.Books.title }}">

cps/templates/index.xml

@@ -1,5 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <feed xmlns="http://www.w3.org/2005/Atom">
+  <icon>{{ url_for('static', filename='favicon.ico') }}</icon>
   <id>urn:uuid:2853dacf-ed79-42f5-8e8a-a7bb3d1ae6a2</id>
   <updated>{{ current_time }}</updated>
   <link rel="self" href="{{url_for('opds.feed_index')}}" type="application/atom+xml;profile=opds-catalog;kind=navigation"/>
@@ -8,7 +9,7 @@
   <link rel="search"
       href="{{url_for('opds.feed_osd')}}"
       type="application/opensearchdescription+xml"/>
-  <link type="application/atom+xml" rel="search" title="{{_('Search')}}" href="{{url_for('opds.feed_cc_search')}}/{searchTerms}" />
+  <link type="application/atom+xml" rel="search" title="{{_('Search')}}" href="{{url_for('opds.feed_normal_search')}}/{searchTerms}" />
   <title>{{instance}}</title>
   <author>
     <name>{{instance}}</name>

cps/templates/layout.html

@@ -37,12 +37,11 @@
           <a class="navbar-brand" href="{{url_for('web.index')}}">{{instance}}</a>
         </div>
         {% if g.current_theme == 1 %}
-            <div class="home-btn"><a class="home-btn-tooltip" href="/" data-toggle="tooltip" title="" data-placement="bottom" data-original-title="Home"></a></div>
+            <div class="home-btn"><a class="home-btn-tooltip" href="{{url_for("web.index",page=1)}}" data-toggle="tooltip" title="" data-placement="bottom" data-original-title="Home"></a></div>
             <div class="plexBack"><a href="{{url_for('web.index')}}"></a></div>
         {% endif %}
         {% if current_user.is_authenticated or g.allow_anonymous %}
-          <form class="navbar-form navbar-left" role="search" action="{{url_for('search.simple_search')}}" method="POST">
-            <input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
+          <form class="navbar-form navbar-left" role="search" action="{{url_for('search.simple_search')}}" method="GET">
             <div class="form-group input-group input-group-sm">
               <label for="query" class="sr-only">{{_('Search')}}</label>
               <input type="text" class="form-control" id="query" name="query" placeholder="{{_('Search Library')}}" value="{{searchterm}}">

cps/templates/list.html

@@ -34,7 +34,7 @@
         <div class="col-xs-10 col-sm-10 col-md-11"><a id="list_{{loop.index0}}" href="{% if entry.format %}{{url_for('web.books_list', data=data, sort_param='stored', book_id=entry.format )}}{% else %}{{url_for('web.books_list', data=data, sort_param='stored', book_id=entry[0].id )}}{% endif %}">
           {% if entry.name %}
           <div class="rating">
-          {% for number in range(entry.name) %}
+          {% for number in range(entry.name|int) %}
             <span class="glyphicon glyphicon-star good"></span>
             {% if loop.last and loop.index < 5 %}
               {% for numer in range(5 - loop.index) %}

cps/templates/osd.xml

@@ -6,7 +6,7 @@
    <Developer>Janeczku</Developer>
    <Contact>https://github.com/janeczku/calibre-web</Contact>
    <Url type="text/html"
-        template="{{url_for('opds.feed_cc_search')}}/{searchTerms}"/>
+        template="{{url_for('opds.feed_normal_search')}}/{searchTerms}"/>
    <Url type="application/atom+xml"
         template="{{url_for('opds.feed_normal_search')}}?query={searchTerms}"/>
    <SyndicationRight>open</SyndicationRight>

cps/templates/readcbr.html

@@ -1,5 +1,6 @@
 <!DOCTYPE html>
 <html>
+
 <head>
   <meta charset="utf-8">
   <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
@@ -20,23 +21,6 @@
   <script src="{{ url_for('static', filename='js/libs/screenfull.min.js') }}"></script>
   <script src="{{ url_for('static', filename='js/compress/uncompress.js') }}"></script>
   <script src="{{ url_for('static', filename='js/kthoom.js') }}"></script>
-  <script>
-    var updateArrows = function() {
-      if ($('input[name="direction"]:checked').val() === "0") {
-        $("#prev_page_key").html("&larr;");
-        $("#next_page_key").html("&rarr;");
-      } else {
-        $("#prev_page_key").html("&rarr;");
-        $("#next_page_key").html("&larr;");
-      }
-    };
-    document.onreadystatechange = function () {
-      if (document.readyState == "complete") {
-      	init("{{ url_for('web.serve_book', book_id=comicfile, book_format=extension) }}");
-      	updateArrows();
-      }
-    }
-  </script>
 </head>
 <body>
 <div id="sidebar">
@@ -77,8 +61,8 @@
   <div id="mainContent" tabindex="-1">
     <div id="mainText" style="display:none"></div>
   </div>
-  <div id="left" class="arrow" onclick="showLeftPage()">‹</div>
-  <div id="right" class="arrow" onclick="showRightPage()">›</div>
+  <div id="left" class="arrow" style="display:none">‹</div>
+  <div id="right" class="arrow" style="display:none">›</div>
 </div>
 
 <div class="modal md-effect-1" id="settings-modal">
@@ -89,8 +73,8 @@
         <table>
           <thead>
           <tr><th colspan="2">{{_('Keyboard Shortcuts')}}</th></tr>
-          </thead>
-          <tbody>
+            </thead>
+            <tbody>
           <tr><td id="prev_page_key">&larr;</td> <td>{{_('Previous Page')}}</td></tr>
           <tr><td id="next_page_key">&rarr;</td> <td>{{_('Next Page')}}</td></tr>
           <tr><td>S</td>      <td>{{_('Single Page Display')}}</td></tr>
@@ -102,21 +86,21 @@
           <tr><td>R</td>      <td>{{_('Rotate Right')}}</td></tr>
           <tr><td>L</td>      <td>{{_('Rotate Left')}}</td></tr>
           <tr><td>F</td>      <td>{{_('Flip Image')}}</td></tr>
-          </tbody>
-        </table>
-      </div>
-      <div class="settings-column">
-        <table id="settings">
-          <thead>
-          <tr>
-            <th>{{_('Settings')}}</th>
-          </tr>
-          </thead>
-          <tbody>
-          <tr>
-            <th>{{_('Theme')}}:</th>
-            <td>
-              <div class="inputs">
+            </tbody>
+          </table>
+        </div>
+        <div class="settings-column">
+          <table id="settings">
+            <thead>
+              <tr>
+                <th>{{_('Settings')}}</th>
+              </tr>
+            </thead>
+            <tbody>
+              <tr>
+                <th>{{_('Theme')}}:</th>
+                <td>
+                  <div class="inputs">
                 <label for="lightTheme"><input type="radio" id="lightTheme" name="theme" value="light" /> {{_('Light')}}</label>
                 <label for="darkTheme"><input type="radio" id="darkTheme" name="theme" value="dark" /> {{_('Dark')}}</label>
               </div>
@@ -139,59 +123,83 @@
                 <label for="fitWidth"><input type="radio" id="fitWidth" name="fitMode" value="87" /> {{_('Width')}}</label>
                 <label for="fitHeight"><input type="radio" id="fitHeight" name="fitMode" value="72" /> {{_('Height')}}</label>
                 <label for="fitNative"><input type="radio" id="fitNative" name="fitMode" value="78" /> {{_('Native')}}</label>
-              </div>
-            </td>
-          </tr>
-          <tr>
-            <th>{{_('Rotate')}}:</th>
-            <td>
-              <div class="inputs">
-                <label for="r0"><input type="radio" id="r0" name="rotateTimes" value="0" /> 0&deg;</label>
-                <label for="r90"><input type="radio" id="r90" name="rotateTimes" value="1" /> 90&deg;</label>
-                <label for="r180"><input type="radio" id="r180" name="rotateTimes" value="2" /> 180&deg;</label>
-                <label for="r270"><input type="radio" id="r270" name="rotateTimes" value="3" /> 270&deg;</label>
-              </div>
-            </td>
-          </tr>
-          <tr>
-            <th>{{_('Flip')}}:</th>
-            <td>
-              <div class="inputs">
-                <label for="vflip"><input type="checkbox" id="vflip" name="vflip" /> {{_('Horizontal')}}</label>
-                <label for="hflip"><input type="checkbox" id="hflip" name="hflip" /> {{_('Vertical')}}</label>
-              </div>
-            </td>
-          </tr>
-          <tr>
-            <th>{{_('Direction')}}:</th>
-            <td>
-              <div class="inputs">
+                  </div>
+                </td>
+              </tr>
+              <tr>
+                <th>{{_('Rotate')}}:</th>
+                <td>
+                  <div class="inputs">
+                    <label for="r0"><input type="radio" id="r0" name="rotateTimes" value="0" /> 0&deg;</label>
+                    <label for="r90"><input type="radio" id="r90" name="rotateTimes" value="1" /> 90&deg;</label>
+                    <label for="r180"><input type="radio" id="r180" name="rotateTimes" value="2" /> 180&deg;</label>
+                    <label for="r270"><input type="radio" id="r270" name="rotateTimes" value="3" /> 270&deg;</label>
+                  </div>
+                </td>
+              </tr>
+              <tr>
+                <th>{{_('Flip')}}:</th>
+                <td>
+                  <div class="inputs">
+                    <label for="vflip"><input type="checkbox" id="vflip" name="vflip" /> {{_('Horizontal')}}</label>
+                    <label for="hflip"><input type="checkbox" id="hflip" name="hflip" /> {{_('Vertical')}}</label>
+                  </div>
+                </td>
+              </tr>
+              <tr>
+                <th>{{_('Direction')}}:</th>
+                <td>
+                  <div class="inputs">
                 <label for="leftToRight"><input type="radio" id="leftToRight" name="direction" value="0" /> {{_('Left to Right')}}</label>
                 <label for="rightToLeft"><input type="radio" id="rightToLeft" name="direction" value="1" /> {{_('Right to Left')}}</label>
               </div>
             </td>
           </tr>
           <tr>
+            <th>{{_('Next Page')}}:</th>
+            <td>
+              <div class="inputs">
+                <label for="resetToTop"><input type="radio" id="resetToTop" name="nextPage" value="0" /> {{_('Reset to Top')}}</label>
+                <label for="rememberPosition"><input type="radio" id="rememberPosition" name="nextPage" value="1" /> {{_('Remember Position')}}</label>
+                  </div>
+                </td>
+              </tr>
+              <tr>                
             <th>{{_('Scrollbar')}}:</th>
             <td>
               <div class="inputs">
                 <label for="showScrollbar"><input type="radio" id="showScrollbar" name="scrollbar" value="1" /> {{_('Show')}}</label>
                 <label for="hideScrollbar"><input type="radio" id="hideScrollbar" name="scrollbar" value="0" /> {{_('Hide')}}</label>
-              </div>
-            </td>
-          </tr>
-          </tbody>
-        </table>
+                  </div>
+                </td>
+              </tr>
+            </tbody>
+          </table>
+        </div>
       </div>
+      <div class="closer icon-cancel-circled"></div>
     </div>
-    <div class="closer icon-cancel-circled"></div>
   </div>
-</div>
-<div class="overlay"></div>
-<script>
-  $('input[name="direction"]').change(function() {
-    updateArrows();
-  });
-</script>
+  <div class="overlay"></div>
+  <input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
+  <script>
+    window.calibre = {
+        bookmarkUrl: "{{ url_for('web.set_bookmark', book_id=comicfile, book_format=extension.upper()) }}",
+        bookmark: "{{ bookmark.bookmark_key if bookmark != None }}",
+        useBookmarks: "{{ current_user.is_authenticated | tojson }}"
+    };
+
+    document.onreadystatechange = function () {
+      if (document.readyState == "complete") {
+          if (calibre.useBookmarks) {
+              currentImage = eval(calibre.bookmark);
+              if (typeof currentImage !== 'number') {
+                  currentImage = 0;
+              }
+          }
+          init("{{ url_for('web.serve_book', book_id=comicfile, book_format=extension) }}");
+      }
+    }
+  </script>
 </body>
 </html>

cps/templates/readdjvu.html

@@ -18,6 +18,6 @@
     <script src="{{ url_for('static', filename='js/reading/djvu_reader.js') }}"></script>
   </head>
   <body>
-	<div id="djvuContainer" file="{{ url_for('web.serve_book', book_id=djvufile, book_format='djvu') }}"></div>
+	<div id="djvuContainer" file="{{ url_for('web.serve_book', book_id=djvufile, book_format=extension) }}"></div>
   </body>
 </html>

cps/templates/search.html

@@ -41,7 +41,7 @@
 
   <div class="row display-flex">
     {% for entry in entries %}
-    <div class="col-sm-3 col-lg-2 col-xs-6 book">
+    <div class="col-sm-3 col-lg-2 col-xs-6 book session">
       <div class="cover">
         {% if entry.Books.has_cover is defined %}
            <a href="{{ url_for('web.show_book', book_id=entry.Books.id) }}" {% if simple==false %}data-toggle="modal" data-target="#bookDetailsModal" data-remote="false"{% endif %}>

cps/templates/search_form.html

@@ -41,7 +41,8 @@
     <div class="form-group">
       <label for="read_status">{{_('Read Status')}}</label>
       <select name="read_status" id="read_status" class="form-control">
-        <option value="" selected></option>
+        <option value="Any" selected>{{_('Any')}}</option>
+        <option value="">{{_('Empty')}}</option>
         <option value="True" >{{_('Yes')}}</option>
         <option value="False" >{{_('No')}}</option>
       </select>

cps/templates/shelf.html

@@ -31,7 +31,7 @@
   {% endif %}
   <div class="row display-flex">
     {% for entry in entries %}
-    <div class="col-sm-3 col-lg-2 col-xs-6 book">
+    <div class="col-sm-3 col-lg-2 col-xs-6 book session">
       <div class="cover">
             <a href="{{ url_for('web.show_book', book_id=entry.Books.id) }}" {% if simple==false %}data-toggle="modal" data-target="#bookDetailsModal" data-remote="false"{% endif %}>
               <span class="img" title="{{entry.Books.title}}" >

cps/templates/shelfdown.html

@@ -19,13 +19,6 @@
        <link href="{{ url_for('static', filename='css/caliBlur.css') }}" rel="stylesheet" media="screen">
        <link href="{{ url_for('static', filename='css/caliBlur_override.css') }}" rel="stylesheet" media="screen">
     {% endif %}
-    <!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
-    <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
-    <!--[if lt IE 9]>
-      <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
-      <script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
-    <![endif]-->
-
     {% block header %}{% endblock %}
   </head>
   <body class="{{ page }} shelf-down">

cps/templates/user_edit.html

@@ -21,7 +21,7 @@
       {% endif %}
         <div class="form-group">
           <label for="password">{{_('Password')}}</label>
-          <input type="password" class="form-control" name="password" id="password" data-lang="{{ current_user.locale }}" data-verify="{{ config.config_password_policy }}" {% if config.config_password_policy %} data-min={{ config.config_password_min_length }} data-special={{ config.config_password_special }} data-upper={{ config.config_password_upper }} data-lower={{ config.config_password_lower }} data-number={{ config.config_password_number }}{% endif %} value="" autocomplete="off">
+          <input type="password" class="form-control" name="password" id="password" data-lang="{{ current_user.locale }}" data-verify="{{ config.config_password_policy }}" {% if config.config_password_policy %} data-min={{ config.config_password_min_length }} data-word={{ config.config_password_character }} data-special={{ config.config_password_special }} data-upper={{ config.config_password_upper }} data-lower={{ config.config_password_lower }} data-number={{ config.config_password_number }}{% endif %} value="" autocomplete="off">
         </div>
     {% endif %}
     <div class="form-group">
@@ -67,7 +67,7 @@
       <div class="btn btn-danger" id="config_delete_kobo_token" data-value="{{ content.id }}" data-remote="false" {% if not content.remote_auth_token.first() %} style="display: none;" {% endif %}>{{_('Delete')}}</div>
     </div>
     <div class="form-group col">
-      <div class="btn btn-default" id="kobo_full_sync" data-value="{{ content.id }}" {% if not content.remote_auth_token.first() %} style="display: none;" {% endif %}>{{_('Force full kobo sync')}}</div>
+      <div class="btn btn-default" id="kobo_full_sync" data-value="{% if current_user.role_admin() %}{{ content.id }}{% else %}0{% endif %}" {% if not content.remote_auth_token.first() %} style="display: none;" {% endif %}>{{_('Force full kobo sync')}}</div>
     </div>
     {% endif %}
     <div class="col-sm-6">
@@ -177,7 +177,7 @@
 <script src="{{ url_for('static', filename='js/libs/bootstrap-table/bootstrap-editable.min.js') }}"></script>
 <script src="{{ url_for('static', filename='js/libs/pwstrength/i18next.min.js') }}"></script>
 <script src="{{ url_for('static', filename='js/libs/pwstrength/i18nextHttpBackend.min.js') }}"></script>
-<script src="{{ url_for('static', filename='js/libs/pwstrength/pwstrength-bootstrap.min.js') }}"></script>
+<script src="{{ url_for('static', filename='js/libs/pwstrength/pwstrength-bootstrap.js') }}"></script>
 <script src="{{ url_for('static', filename='js/password.js') }}"></script>
 <script src="{{ url_for('static', filename='js/table.js') }}"></script>
 {% endblock %}

cps/tornado_wsgi.py

@@ -16,12 +16,12 @@
 #  You should have received a copy of the GNU General Public License
 #  along with this program. If not, see <http://www.gnu.org/licenses/>.
 
-
 from tornado.wsgi import WSGIContainer
 import tornado
 
 from tornado import escape
 from tornado import httputil
+from tornado.ioloop import IOLoop
 
 from typing import List, Tuple, Optional, Callable, Any, Dict, Text
 from types import TracebackType
@@ -34,61 +34,67 @@ if typing.TYPE_CHECKING:
 class MyWSGIContainer(WSGIContainer):
 
     def __call__(self, request: httputil.HTTPServerRequest) -> None:
-        data = {}  # type: Dict[str, Any]
-        response = []  # type: List[bytes]
+        if tornado.version_info < (6, 3, 0, -99):
+            data = {}  # type: Dict[str, Any]
+            response = []  # type: List[bytes]
 
-        def start_response(
-            status: str,
-            headers: List[Tuple[str, str]],
-            exc_info: Optional[
-                Tuple[
-                    "Optional[Type[BaseException]]",
-                    Optional[BaseException],
-                    Optional[TracebackType],
-                ]
-            ] = None,
-        ) -> Callable[[bytes], Any]:
-            data["status"] = status
-            data["headers"] = headers
-            return response.append
+            def start_response(
+                status: str,
+                headers: List[Tuple[str, str]],
+                exc_info: Optional[
+                    Tuple[
+                        "Optional[Type[BaseException]]",
+                        Optional[BaseException],
+                        Optional[TracebackType],
+                    ]
+                ] = None,
+            ) -> Callable[[bytes], Any]:
+                data["status"] = status
+                data["headers"] = headers
+                return response.append
 
-        app_response = self.wsgi_application(
-            MyWSGIContainer.environ(request), start_response
-        )
+            app_response = self.wsgi_application(
+                MyWSGIContainer.environ(self, request), start_response
+            )
+            try:
+                response.extend(app_response)
+                body = b"".join(response)
+            finally:
+                if hasattr(app_response, "close"):
+                    app_response.close()  # type: ignore
+            if not data:
+                raise Exception("WSGI app did not call start_response")
+
+            status_code_str, reason = data["status"].split(" ", 1)
+            status_code = int(status_code_str)
+            headers = data["headers"]  # type: List[Tuple[str, str]]
+            header_set = set(k.lower() for (k, v) in headers)
+            body = escape.utf8(body)
+            if status_code != 304:
+                if "content-length" not in header_set:
+                    headers.append(("Content-Length", str(len(body))))
+                if "content-type" not in header_set:
+                    headers.append(("Content-Type", "text/html; charset=UTF-8"))
+            if "server" not in header_set:
+                headers.append(("Server", "TornadoServer/%s" % tornado.version))
+
+            start_line = httputil.ResponseStartLine("HTTP/1.1", status_code, reason)
+            header_obj = httputil.HTTPHeaders()
+            for key, value in headers:
+                header_obj.add(key, value)
+            assert request.connection is not None
+            request.connection.write_headers(start_line, header_obj, chunk=body)
+            request.connection.finish()
+            self._log(status_code, request)
+        else:
+            IOLoop.current().spawn_callback(self.handle_request, request)
+
+
+    def environ(self, request: httputil.HTTPServerRequest) -> Dict[Text, Any]:
         try:
-            response.extend(app_response)
-            body = b"".join(response)
-        finally:
-            if hasattr(app_response, "close"):
-                app_response.close()  # type: ignore
-        if not data:
-            raise Exception("WSGI app did not call start_response")
-
-        status_code_str, reason = data["status"].split(" ", 1)
-        status_code = int(status_code_str)
-        headers = data["headers"]  # type: List[Tuple[str, str]]
-        header_set = set(k.lower() for (k, v) in headers)
-        body = escape.utf8(body)
-        if status_code != 304:
-            if "content-length" not in header_set:
-                headers.append(("Content-Length", str(len(body))))
-            if "content-type" not in header_set:
-                headers.append(("Content-Type", "text/html; charset=UTF-8"))
-        if "server" not in header_set:
-            headers.append(("Server", "TornadoServer/%s" % tornado.version))
-
-        start_line = httputil.ResponseStartLine("HTTP/1.1", status_code, reason)
-        header_obj = httputil.HTTPHeaders()
-        for key, value in headers:
-            header_obj.add(key, value)
-        assert request.connection is not None
-        request.connection.write_headers(start_line, header_obj, chunk=body)
-        request.connection.finish()
-        self._log(status_code, request)
-
-    @staticmethod
-    def environ(request: httputil.HTTPServerRequest) -> Dict[Text, Any]:
-        environ = WSGIContainer.environ(request)
+            environ = WSGIContainer.environ(self, request)
+        except TypeError as e:
+            environ = WSGIContainer.environ(request)
         environ['RAW_URI'] = request.path
         return environ