MySQL Flask sqlalchemy.exc.IntegrityError

Installation und Anwendung von Datenbankschnittstellen wie SQLite, PostgreSQL, MariaDB/MySQL, der DB-API 2.0 und sonstigen Datenbanksystemen.
Antworten
snowandsun
User
Beiträge: 1
Registriert: Montag 13. November 2023, 14:39

Guten Tag Zusammen
Ich wäre froh wenn mir jemand mit folgendem Problem weiterhelfen könnte:
Python code to save data in mySQL:
Der Code und die mySQL DB befinden sich auf Pythonanywhere. Die Funktion sollte ein JSON Objekt in einer mySQL DB specichern. Gelegentlich (nicht immer) erhalte ich folgende Fehlermeldung:

Error message: sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; Duplicate entry '0' for key 'revisionsTable.PRIMARY'

Das komplette Error log und Server log weiter unten.
Im Error log heisst es in könnte den Autoflush ausschalten ich möchte aber gerne die Ursache des Problems verstehen.
Der Primary Key 1 fehlt wenn ich:

im MySQL Bash: select * FROM revisionsTable;

| 0 | initial | Initial revision (Primary Key 0

| 2 | ea6db717d0ca47d9cc4e1860ae9de6711 | | NULL | {

Code: Alles auswählen

@saveRevisions_bp.route('/saveRevisionData', methods=['POST'])
def save_revision_data():
    data = request.json
    print("Raw request data:", data)
    revisions_data = data.get('revisionsData')
    print("Revisions Data Received:", revisions_data)
    updated_revisions = []

    if not revisions_data:
        return jsonify({"error": "No revisions data provided"}), 400

    json_fields = {'diffData', 'attributes', 'authorsIds'}

    for rev_data in revisions_data:
        revision_id = rev_data.get('id')

        revision = Revisions.query.filter_by(revisionId=revision_id).first()

        if revision:
            for key, value in rev_data.items():
                if key in json_fields:
                    setattr(revision, key, json.dumps(value))
                else:
                    setattr(revision, key, value)
        else:
            new_revision_data = {k: json.dumps(v) if k in json_fields else v for k, v in rev_data.items()}
            new_revision_data['revisionId'] = revision_id
            new_revision_data['createdAt'] = datetime.utcnow()
            new_revision = Revisions(**new_revision_data)
            db.session.add(new_revision)
            revision = new_revision

        # Check if createdAt is a string and convert it to datetime if necessary
        created_at = revision.createdAt
        if isinstance(created_at, str):
            try:
                created_at = parser.parse(created_at)
            except ValueError:
                created_at = None  # or set a default value

        updated_revisions.append({
            "id": revision.id,
            "createdAt": created_at.isoformat() if created_at else None
        })

        print("Saving revision:", revision.id, "with attributes:", revision.attributes)

    db.session.commit()
    return jsonify(updated_revisions)
MySQL DB Config:

Code: Alles auswählen

[from flask import Flask
from flask_migrate import Migrate

from models import db
from flask_cors import CORS

app = Flask(__name__)
app.config["DEBUG"] = True

# Enable CORS for all routes
CORS(app)

SQLALCHEMY_DATABASE_URI = "mysql+mysqlconnector://{username}:{password}@{hostname}/{databasename}".format(
    username=„myusername,
    password=„mypassword,
    hostname=„myusername.mysql.pythonanywhere-services.com",
    databasename=„myusername$revisions",
)
app.config["SQLALCHEMY_DATABASE_URI"] = SQLALCHEMY_DATABASE_URI
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False

# Engine options configuration
SQLALCHEMY_ENGINE_OPTIONS = {
    'pool_recycle': 299,
    'pool_pre_ping': True
}
app.config["SQLALCHEMY_ENGINE_OPTIONS"] = SQLALCHEMY_ENGINE_OPTIONS


db.init_app(app)
migrate = Migrate(app, db)

app.register_blueprint(getRevisions_bp, url_prefix='/')
app.register_blueprint(saveRevisions_bp, url_prefix='/')
app.register_blueprint(fetchRevisions_bp, url_prefix='/')/code]

Pythonanywhere Error log:
Error message: sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; Duplicate entry '0' for key 'revisionsTable.PRIMARY'

2023-11-13 13:17:22,887: Error running WSGI application
2023-11-13 13:17:22,964: sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
2023-11-13 13:17:22,964: (mysql.connector.errors.IntegrityError) 1062 (23000): Duplicate entry '0' for key 'revisionsTable.PRIMARY'
2023-11-13 13:17:22,964: [SQL: UPDATE `revisionsTable` SET id=%(id)s, `diffData`=%(diffData)s, `createdAt`=%(createdAt)s, `fromVersion`=%(fromVersion)s, `toVersion`=%(toVersion)s WHERE `revisionsTable`.id = %(revisionsTable_id)s]
2023-11-13 13:17:22,964: [parameters: {'id': 'e2019bd011466e942e7feda31a4624579', 'diffData': '{"main": {"insertions": "[{\\"type\\":\\"c\\",\\"name\\":\\"p\\",\\"attributes\\":[],\\"children\\":[\\"Test dritte\\",{\\"type\\":\\"a\\",\\"name\\" ... (3596 characters truncated) ... \\"type\\":\\"c\\",\\"name\\":\\"p\\",\\"attributes\\":[],\\"children\\":[]}]", "attachChange": null, "attributesBefore": {}, "attributesAfter": {}}}', 'createdAt': '2023-11-13T13:17:22.487Z', 'fromVersion': 37, 'toVersion': 37, 'revisionsTable_id': 4}]
2023-11-13 13:17:22,964: (Background on this error at: https://sqlalche.me/e/20/gkpj)
2023-11-13 13:17:22,965:   File "/home/adrianruchti/.virtualenvs/Flask_Apps/lib/python3.10/site-packages/flask/app.py", line 1478, in __call__

Pythonanywhere Server log:


2023-11-13 13:15:55 Raw request data: {'revisionsData': [{'diffData': {'main': {'insertions': '[{"type":"c","name":"p","attributes":[],"children":["Test dritte",{"type":"a","name":"strong","attributes":[],"children":[" "]},{"type":"a","name":"i","attributes":[],"children":["Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test vierte Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test "]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test neue Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test zusätzliche Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test 3"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","
2023-11-13 13:15:55 Revisions Data Received: [{'diffData': {'main': {'insertions': '[{"type":"c","name":"p","attributes":[],"children":["Test dritte",{"type":"a","name":"strong","attributes":[],"children":[" "]},{"type":"a","name":"i","attributes":[],"children":["Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test vierte Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test "]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test neue Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test zusätzliche Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test 3"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes"
2023-11-13 13:15:55 Saving revision: initial with attributes: None
2023-11-13 13:15:55 Saving revision: e2019bd011466e942e7feda31a4624579 with attributes: None
2023-11-13 13:17:22 Raw request data: {'revisionsData': [{'diffData': {'main': {'insertions': '[{"type":"c","name":"p","attributes":[],"children":["Test dritte",{"type":"a","name":"strong","attributes":[],"children":[" "]},{"type":"a","name":"i","attributes":[],"children":["Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test vierte Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test "]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test neue Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test zusätzliche Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test 3"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","
2023-11-13 13:17:22 Revisions Data Received: [{'diffData': {'main': {'insertions': '[{"type":"c","name":"p","attributes":[],"children":["Test dritte",{"type":"a","name":"strong","attributes":[],"children":[" "]},{"type":"a","name":"i","attributes":[],"children":["Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test vierte Revision"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test "]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test neue Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test zusätzliche Zeile"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes":[],"children":["Test 3"]}]},{"type":"c","name":"p","attributes":[],"children":[{"type":"a","name":"i","attributes"
2023-11-13 13:17:22 Saving revision: e2019bd011466e942e7feda31a4624579 with attributes: None

Vielen Dank für Eure Hilfe.
snowandsun
Sirius3
User
Beiträge: 17754
Registriert: Sonntag 21. Oktober 2012, 17:20

Was ist der Unterschied zwischen `rev_data` und `revisions_data`? Es kann gut sein, dass Du das in zwei Wochen nicht mehr weißt. Deshalb benutze sprechende Variablennamen ohne kryptische Abkürzungen. Das Wort `data` ist so generisch, dass man es weglassen kann.
Unbekannte Daten aus irgendeinem Web-Aufruf sollte man nicht ungesehen verarbeiten. Was Du da per setattr an eine Revisions-Instanz bindet, hat der Angreifer in der Hand. Deshalb explizit angeben, was da gesetzt werden soll.
Es ist meist auch keine gute Idee, wenn man von außen eine interne Datenbank-ID vorgeben kann, damit hast Du ja jetzt auch Probleme. Also lass das setzen der ID einfach weg.
Benutzeravatar
__blackjack__
User
Beiträge: 13116
Registriert: Samstag 2. Juni 2018, 10:21
Wohnort: 127.0.0.1
Kontaktdaten:

Zum validieren und eventuell auch umwandeln von so etwas wie JSON-Daten gibt es auch diverse Bibliotheken. Cerberus, Marshmallow, glom, …
„All religions are the same: religion is basically guilt, with different holidays.” — Cathy Ladman
Antworten