Compare commits

..
17 Commits
Author SHA1 Message Date
abdus de6771ac86 feat(sync-alfred): Add WebDAV preference sync 2026-09-05 09:29:46 +02:00
abdus f3c20d8ed5 feat(export-pg): Add batched Postgres to SQLite export 2026-09-05 09:29:29 +02:00
abdus 3e5d58ea25 feat(backup-to-external): Add hardlink-aware backup tool 2026-09-05 09:29:26 +02:00
abdus 5a3cbdde6a fix(sort-images): Detect formats from file signatures 2026-09-05 09:29:21 +02:00
abdus 256b81c828 fix(rename-children): Remove empty directories 2026-09-05 09:29:19 +02:00
abdus 8c78f79720 fix(mpv-pip): Accept newline-separated targets 2026-09-05 09:29:17 +02:00
abdus 5216362be7 chore(file-renamer): Use the default Python interpreter 2026-09-05 09:29:11 +02:00
abdus 69df91380c fix(download-imageset): Flatten downloaded directories 2026-09-05 09:29:08 +02:00
abdus 7ede67b98f feat(ai-photorealistic): Update photorealistic prompt 2026-09-05 09:29:04 +02:00
abdus d5cf81a919 feat(chat): Support Youtube transcripts 2026-08-11 09:51:30 +02:00
abdus 3cba11f1df feat(snot): Add gridvid support, update Alfred integrations 2026-08-11 09:50:55 +02:00
abdus ec07e27924 fix(chat): Update models 2026-08-09 14:39:43 +02:00
abdus 3b4fc7159e feat(chat): Add Clear button to history popup
Lets you wipe all saved conversations from localStorage (not just
today's) after a confirmation prompt.
2026-07-28 10:13:00 +02:00
abdus eee4addd86 feat(chat): Add undo button to trim conversation to a prior message
Lets you click Undo under a user message to drop it and everything
after it, refilling the prompt with that message's text/images so it
can be edited and resent.
2026-07-28 09:22:15 +02:00
abdus 0ddf2c4db5 refactor(download_imageset): Migrate CLI from argparse to click 2026-07-28 08:41:40 +02:00
abdus c9228ae4a7 feat(youtube_transcript_v2): Add --language/--auto options and unwrap output text
Auto mode prioritizes de/en/tr and falls back to the first available
transcript language; a shorter error is shown when the default fetch fails.
2026-07-26 11:08:02 +02:00
Abdüssamet Kocak 051b7b12f0 feat: Add MPV PIP 2026-07-25 09:09:40 +02:00
12 changed files with 1445 additions and 176 deletions
+6 -65
View File
@@ -27,65 +27,6 @@ MODEL_ID = "flux-2-klein-9b"
# MODEL_ID = "qwen-image"
SYSTEM_PROMPT = """Make this incredibly photorealistic.
Highly stylized, striking, highly detailed photo of a young, lithe woman.
Age her by 5 years and make her a 20 year old, taller, slimmer, more slender, fitter version of herself.
Make her 6ft tall.
Give her realistic spotless skin texture.
Her face skin must also be realistic, without any makeup.
Make the background realistic and consistent with the lighting on her.
Give her an angular, not round, narrow, elongated neck and long face shape defined by a structured, clean jawline that tapers toward a firm, slightly rounded chin.
Do not make the head larger than the body, keep them in perfect proportion.
Give her high, prominent cheekbones giving her sculpted, chieseled face that taper down to a more narrow, delicate chin.
Make her really pretty and seductive.
Preserve unnatural lip colors and wetness.
Do not change the facial expression, emotion, hand and body pose, mouth, tongue and eyelid position.
Lower her eyes slightly to give her an alluring look.
Mouth slightly agape giving her a seductive look.
Give her a sharp, cunning gaze, a very slight pleasant smirk without changing her head pose.
Remove the drool.
Keep the pose and don't turn people around."""
SYSTEM_PROMPT = """
Turn this into a heavily stylized real life photo of a woman.
Age her by 5 years and make her a 20 year old, taller, slimmer, more slender, fitter version of herself.
Apply realistic textures without grain or noise.
Skin should be smooth, flawless, without pores, clean and without blemishes.
Keep the shine.
Make her taller, slimmer, slender. Don't make the head shorter.
Do not change the pose, gaze, emotion.
Her lips are parted, creating a soft, seductive expression.
"""
SYSTEM_PROMPT = """
restyle this as an edited, finished photo taken by sony a7 iv.
Photorealistic textures.
same color grading, same colors and lighting.
Realistic, flawless skin.
White people should have a pale skin.
keep the same ethnicity and facial features.
Make the children taller and more slender with slightly elongated feminine neck and higher cheekbones, give them smaller, slimmer more feminine head and tapered face; round, smooth chin.
age them by 5 years and make them look like 20 year old adults.
keep everything else about her the same.
same facial + eye expression (keep them as open as in the original), same pose and emotion, same tongue position.
same head and eye angle.
do not turn her face around.
same hair style, color, length.
same composition and crop.
keep the same makeup, do not change the colors of the lips, keep them glossy and wet.
"""
SYSTEM_PROMPT = """
Artistic super photorealistic conversion.
extend the background horizontally but keep the people.
@@ -98,27 +39,27 @@ High dynamic range with a focus on rich textures.
balanced exposure.
same color grading and LUT.
remove compression artifacts.
remove noise and compression artifacts.
she has spotless, supple, shiny skin without splotches.
#pale skin, goth make-up.
#reduce musculature.
#keep the original facial expression, emotion, hand and body pose.
make her prettier, give her hourglass figure, a satisfied look, voluminus hair.
# make her prettier, give her hourglass figure, a satisfied look, voluminus hair.
parted lips, seductive gaze, subtly lowered eyelids.
make her face look like a 25 year old, raised cheekbones, tapered face shape, pointy chin.
#parted lips, seductive gaze, subtly lowered eyelids.
#make her face look like a 25 year old, raised cheekbones, tapered face shape, pointy chin.
#avoid making her look like a child and maintain adult proportions.
#give her a subtle mischievous, confident smile, avoid neutral face.
#make her face slimmer.
preserve the original skin color.
do not change the colors of the lips, keep them wet.
# do not change the colors of the lips, keep them wet.
remove all logos and watermarks.
#reflective outfit,
#glossy, reflective outfit,
#clingy, skin-tight clothes.
clingy, skin-tight clothes.
"""
+319
View File
@@ -0,0 +1,319 @@
#!/usr/bin/env python3
"""Mark files for backup, then copy them to an external drive and remove
all their hardlinked siblings once the copy is verified.
Usage:
backup_tool.py mark <file> [file...] stage a file for backup
backup_tool.py unmark <file> [file...] drop a file from staging
backup_tool.py list show what's staged
backup_tool.py run [--yes] copy staged files to the drive,
then delete originals + siblings
Config (override via environment):
BACKUP_STAGING_DIR default: ~/.backup-staging
Destination routing is controlled by the RULES table below — edit it to
change which drive/subdirectory a file backs up to based on its source path
and type.
"""
import argparse
import hashlib
import os
import shutil
import subprocess
import sys
from collections.abc import Callable
from dataclasses import dataclass, field
from pathlib import Path
STAGING_DIR = Path(
os.environ.get("BACKUP_STAGING_DIR", str(Path.home() / ".backup-staging"))
)
VIDEO_EXTS = {".mp4", ".mov", ".mkv", ".avi", ".m4v", ".webm"}
# Roots under which an external drive's mount point may appear. Native
# drives (HFS+/APFS) mount under /Volumes; Mounty mounts NTFS drives
# read-write under ~/.mounty instead.
MOUNT_ROOTS = [Path("/Volumes"), Path.home() / ".mounty"]
def is_video(path: Path) -> bool:
return path.suffix.lower() in VIDEO_EXTS
@dataclass
class Rule:
prefix: Path # source files under this path are matched
volume: str # drive name as it appears under a mount root, e.g. "FIVER"
subdir: str # path under the volume root to copy into, e.g. "_ingress"
match_file: Callable[[Path], bool] | None = field(
default=None
) # None matches any file
def candidate_mount_points(self) -> list[Path]:
"""Where this volume might be mounted: a regular /Volumes mount, or
a Mounty read-write remount under ~/.mounty."""
return [root / self.volume for root in MOUNT_ROOTS]
def mounted_at(self) -> Path | None:
"""Whichever candidate mount point is actually mounted right now, or None."""
for mount_point in self.candidate_mount_points():
if is_mounted(mount_point):
return mount_point
return None
def matches(self, path: Path) -> bool:
if not path.is_relative_to(self.prefix):
return False
if self.match_file is not None and not self.match_file(path):
return False
return True
def destination_for(self, path: Path, mount_point: Path) -> Path:
rel = path.relative_to(self.prefix)
return mount_point / self.subdir / rel
# Order matters: first matching rule wins.
RULES = [
Rule(
prefix=Path.home() / "Downloads/temp",
volume="FIVER",
subdir="_ingress",
match_file=is_video,
),
Rule(
prefix=Path.home() / "Movies",
volume="FAST",
subdir="_movies",
),
]
def rule_for(path: Path) -> Rule | None:
for rule in RULES:
if rule.matches(path):
return rule
return None
def die(msg: str) -> None:
print(f"error: {msg}", file=sys.stderr)
sys.exit(1)
def staged_path_for(abs_path: Path) -> Path:
return STAGING_DIR / abs_path.relative_to(abs_path.anchor)
def original_path_for(staged_path: Path) -> Path:
return Path("/") / staged_path.relative_to(STAGING_DIR)
def sha256(path: Path) -> str:
h = hashlib.sha256()
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def mountpoint_of(path: Path) -> str:
out = subprocess.run(
["df", str(path)], capture_output=True, text=True, check=True
).stdout
return out.strip().splitlines()[-1].split()[-1]
def mount_line(mount_point: Path) -> str | None:
out = subprocess.run(["mount"], capture_output=True, text=True, check=True).stdout
for line in out.splitlines():
if f" on {mount_point} " in line:
return line
return None
def is_mounted(mount_point: Path) -> bool:
return mount_line(mount_point) is not None
def is_read_only(mount_point: Path) -> bool:
"""True if mounted read-only — the state NTFS drives auto-mount into
before Mounty remounts them read-write via ntfs-3g."""
line = mount_line(mount_point)
return line is not None and "read-only" in line
def find_siblings(path: Path) -> list[Path]:
"""All paths sharing this file's inode, scoped to its own filesystem."""
inode = path.stat().st_ino
root = mountpoint_of(path)
result = subprocess.run(
["find", root, "-xdev", "-inum", str(inode)],
capture_output=True,
text=True,
)
return [Path(p) for p in result.stdout.splitlines() if p]
def cmd_mark(args: argparse.Namespace) -> None:
if not args.files:
die("usage: mark <file> [file...]")
for f in args.files:
p = Path(f)
if not p.is_file():
print(f"skip (not a regular file): {f}", file=sys.stderr)
continue
abs_path = p.resolve()
if rule_for(abs_path) is None:
die(f"no backup rule matches '{abs_path}' — add one to RULES")
dest = staged_path_for(abs_path)
dest.parent.mkdir(parents=True, exist_ok=True)
if dest.exists():
print(f"already marked: {abs_path}")
continue
try:
os.link(abs_path, dest)
except OSError as e:
die(
f"could not hardlink '{abs_path}' into staging "
f"(staging dir must be on the same filesystem as the file): {e}"
)
print(f"marked: {abs_path}")
def cmd_unmark(args: argparse.Namespace) -> None:
if not args.files:
die("usage: unmark <file> [file...]")
for f in args.files:
abs_path = Path(f).resolve()
staged = staged_path_for(abs_path)
if staged.exists():
staged.unlink()
print(f"unmarked: {abs_path}")
else:
print(f"not marked: {abs_path}", file=sys.stderr)
def cmd_list(_args: argparse.Namespace) -> None:
if not STAGING_DIR.is_dir():
print("(nothing staged)")
return
staged = sorted(p for p in STAGING_DIR.rglob("*") if p.is_file())
if not staged:
print("(nothing staged)")
return
for p in staged:
print("/" + str(p.relative_to(STAGING_DIR)))
def cmd_run(args: argparse.Namespace) -> None:
if not STAGING_DIR.is_dir():
print("(nothing staged)")
return
staged = sorted(p for p in STAGING_DIR.rglob("*") if p.is_file())
if not staged:
print("(nothing staged)")
return
# Resolve each staged file's destination; skip (with a message) anything
# whose target rule can no longer be matched or whose volume isn't mounted.
plan = []
for src in staged:
original = original_path_for(src)
rule = rule_for(original)
if rule is None:
print(f"skipping (no rule matches): {original}", file=sys.stderr)
continue
mount_point = rule.mounted_at()
if mount_point is None:
print(f"skipping (drive '{rule.volume}' not mounted): {original}")
continue
if is_read_only(mount_point):
print(
f"skipping (drive '{rule.volume}' is mounted read-only — open Mounty "
f"and choose 'Remount' to enable write access): {original}"
)
continue
dest = rule.destination_for(original, mount_point)
plan.append((src, dest))
if not plan:
print("nothing to do — connect the relevant drive(s) and run again.")
return
print("The following staged files will be copied to their destinations,")
print(
"then ALL hardlinked copies of each (including the originals) will be deleted:"
)
for _src, dest in plan:
print(f" -> {dest}")
print()
if not args.yes:
reply = input("Proceed? [y/N] ").strip().lower()
if reply not in ("y", "yes"):
print("aborted.")
sys.exit(1)
for src, dest in plan:
dest.parent.mkdir(parents=True, exist_ok=True)
print(f"copying: {original_path_for(src)} -> {dest}")
shutil.copy2(src, dest)
if sha256(src) != sha256(dest):
print(
f" verification FAILED, leaving originals in place: {original_path_for(src)}",
file=sys.stderr,
)
continue
siblings = find_siblings(src)
print(f" verified. removing {len(siblings)} hardlinked path(s):")
for sib in siblings:
print(f" {sib}")
for sib in siblings:
try:
sib.unlink()
except OSError as e:
print(f" failed to remove {sib}: {e}", file=sys.stderr)
# Clean up now-empty staging directories.
for dirpath, dirnames, filenames in os.walk(STAGING_DIR, topdown=False):
d = Path(dirpath)
if d != STAGING_DIR and not any(d.iterdir()):
d.rmdir()
print("done.")
def main() -> None:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
sub = parser.add_subparsers(dest="command", required=True)
p_mark = sub.add_parser("mark")
p_mark.add_argument("files", nargs="+")
p_mark.set_defaults(func=cmd_mark)
p_unmark = sub.add_parser("unmark")
p_unmark.add_argument("files", nargs="+")
p_unmark.set_defaults(func=cmd_unmark)
p_list = sub.add_parser("list")
p_list.set_defaults(func=cmd_list)
p_run = sub.add_parser("run")
p_run.add_argument("--yes", action="store_true", help="skip confirmation prompt")
p_run.set_defaults(func=cmd_run)
args = parser.parse_args()
args.func(args)
if __name__ == "__main__":
main()
+282 -31
View File
@@ -74,6 +74,49 @@
overflow-y: auto;
padding-right: 0.5rem;
}
.attachment-card {
max-width: 85%;
margin: 1rem 0;
padding: 0.8rem 1.2rem;
border: 1px solid var(--border-color);
border-radius: 0.8rem;
background: var(--bg-secondary);
}
.attachment-card__title {
font-weight: 700;
}
.attachment-card__meta {
color: var(--text-secondary);
font-size: 0.9em;
}
.attachment-card summary {
cursor: pointer;
margin-top: 0.5rem;
}
.attachment-card pre {
max-height: 14rem;
overflow: auto;
white-space: pre-wrap;
overflow-wrap: anywhere;
padding: 0.6rem;
background: var(--bg-code);
border: 1px solid var(--border-color);
border-radius: 0.3rem;
}
.suggested-questions {
display: flex;
flex-wrap: wrap;
gap: 0.5rem;
margin: 0.75rem 0;
}
.suggested-questions button {
cursor: pointer;
padding: 0.35rem 0.6rem;
color: var(--text-primary);
background: var(--bg-primary);
border: 1px solid var(--border-color);
border-radius: 0.3rem;
}
.message {
padding: 0.8rem 1.2rem;
border-radius: 0.8rem;
@@ -97,6 +140,23 @@
margin-left: auto;
white-space: pre-wrap;
}
.undo-btn {
display: block;
margin-left: auto;
margin-top: 0.4rem;
background: rgba(255, 255, 255, 0.15);
color: var(--text-user);
border: 1px solid rgba(255, 255, 255, 0.4);
border-radius: 0.3rem;
padding: 0.15rem 0.5rem;
font-size: 0.8em;
cursor: pointer;
opacity: 0.7;
transition: opacity 0.2s;
}
.undo-btn:hover {
opacity: 1;
}
.system-message {
background: var(--bg-secondary);
margin-right: auto;
@@ -230,6 +290,22 @@
.history-item:hover {
background: var(--hover-bg);
}
.clear-history-btn {
display: block;
width: 100%;
margin-top: 0.5rem;
padding: 0.3rem;
background: var(--bg-secondary);
color: var(--text-primary);
border: 1px solid var(--border-color);
border-radius: 0.3rem;
font-size: 0.9em;
cursor: pointer;
transition: background-color 0.2s;
}
.clear-history-btn:hover {
background: var(--hover-bg);
}
.history-time {
font-weight: 700;
color: var(--text-secondary);
@@ -275,6 +351,21 @@
<div class="chat-container">
<!-- Messages -->
<div class="messages" x-ref="msgBox" @click="onClickMessage" @scroll="handleScroll">
<section class="attachment-card" x-show="attachment">
<div class="attachment-card__title">Transcript attached</div>
<div class="attachment-card__meta" x-text="`${attachment?.length.toLocaleString()} characters`"></div>
<details>
<summary>Show transcript</summary>
<pre x-text="attachment?.value"></pre>
</details>
<template x-if="visibleMessages.length === 0 && !isBusy">
<div class="suggested-questions">
<button @click="useSuggestedQuestion('What are the key points?')">Key points</button>
<button @click="useSuggestedQuestion('What are the most important quotes?')">Notable quotes</button>
<button @click="useSuggestedQuestion('What should I challenge or verify?')">What to verify</button>
</div>
</template>
</section>
<template x-for="(msg, idx) in visibleMessages" :key="idx">
<div :class="msg.role === 'user' ? 'message user-message' : 'message system-message'">
<template x-if="msg.role === 'user'">
@@ -283,6 +374,7 @@
<img :src="img.dataUrl" style="max-width: 100%; border-radius: 0.3rem; display: block; margin-bottom: 0.3rem;" />
</template>
<div class="text" x-text="msg.content"></div>
<button class="undo-btn" title="Trim conversation back to before this message" @click="trimTo(msg)">Undo</button>
</div>
</template>
<template x-if="msg.role === 'assistant'">
@@ -295,7 +387,7 @@
<!-- Textarea -->
<div class="prompt-wrapper">
<div x-show="pendingImages.length > 0" style="display: flex; gap: 0.5rem; flex-wrap: wrap; align-items: flex-start;">
<div x-show="!attachment && pendingImages.length > 0" style="display: flex; gap: 0.5rem; flex-wrap: wrap; align-items: flex-start;">
<template x-for="(img, i) in pendingImages" :key="img.dataUrl">
<div style="position: relative; display: inline-block;">
<img :src="img.dataUrl" style="max-height: 72px; max-width: 120px; object-fit: cover; border-radius: 0.3rem; display: block;" />
@@ -319,7 +411,7 @@
<button class="submit" :class="isBusy && 'busy'" @click="isBusy ? abort() : handleSubmit()" x-text="isBusy ? 'Cancel' : 'Send'"></button>
<!-- Model Selector -->
<details id="model-selector">
<details id="model-selector" x-show="!attachment">
<summary x-text="'Model: ' + activeModelName"></summary>
<template x-for="m in models" :key="m.name">
<label class="model-opt__label" style="display: block">
@@ -330,7 +422,7 @@
</details>
<!-- History -->
<details id="history-panel" x-ref="historyPanel">
<details id="history-panel" x-ref="historyPanel" x-show="!attachment">
<summary>History</summary>
<div class="history-list">
<template x-if="historyItems.length === 0">
@@ -342,6 +434,9 @@
<span class="history-excerpt" x-text="getExcerpt(item.messages)"></span>
</div>
</template>
<template x-if="historyItems.length > 0">
<button class="clear-history-btn" @click="clearHistory">Clear</button>
</template>
</div>
</details>
</div>
@@ -355,18 +450,67 @@
You are a helpful AI assistant.
- Do not preach about implications, do not praise excessively.
- Keep a critical mindset, do not assume blindly.
- Answer in valid markdown only, no exceptions.
- When I ask something about code, or software development, keep in mind that I am a senior backend developer (but do not mention that explicitly), and do not explain excessively, but and challenge my statements with better alternatives if I don't sound right, especially in coding. Do not bring it up otherwise.
- When I ask something about code, or software development, keep in mind that I am a senior backend developer (but do not mention that explicitly), and do not explain excessively, but and challenge my statements where needed with better alternatives if I don't sound right, especially in coding. Do not bring it up otherwise.
- Do not give me code if the topic is not about programming or software development.
`
.replace(/^\s+/gm, "")
.trim();
const defaultVideoSummaryPrompt = "Summarize this video or transcript. Cover the key points, necessary context, and notable quotes. Skip promotions.";
function getTranscriptAttachment(transcript) {
const value = typeof transcript === "string" ? transcript.trim() : "";
if (value) return { kind: "text", value, length: value.length };
return null;
}
function getSystemInstruction(attachment) {
if (!attachment) return systemPrompt;
const attachmentRules = `
You are answering questions about the attached transcript.
Base your answers on that source. State when the source does not establish an answer, do not invent details, and preserve exact wording when the user asks for quotes.
`
.replace(/^\s+/gm, "")
.trim();
return `${attachmentRules}\n\nTranscript:\n${attachment.value}`;
}
const initialAttachment = getTranscriptAttachment(window.env.YOUTUBE_TRANSCRIPT);
const initialPromptText = window.env.PROMPT ? `${window.env.PROMPT}\n\n` : "";
const geminiCachePromises = new Map();
async function getGeminiAttachmentCache({ model, apiKey, systemInstruction, attachment }) {
if (!attachment) return null;
const key = `${model}:${attachment.kind}:${attachment.value}`;
if (!geminiCachePromises.has(key)) {
geminiCachePromises.set(
key,
fetch(`https://generativelanguage.googleapis.com/v1beta/cachedContents?key=${apiKey}`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model: `models/${model}`,
systemInstruction: { parts: [{ text: systemInstruction }] },
contents: [],
ttl: "3600s",
}),
})
.then(async (response) => (response.ok ? response.json() : null))
.then((cache) => cache?.name || null)
.catch(() => null)
);
}
return geminiCachePromises.get(key);
}
const chatApp = () => ({
messages: [{ role: "system", content: systemPrompt }],
pendingImages: [],
promptText: window.env.PROMPT ? `${window.env.PROMPT}\n\n` : "",
attachment: initialAttachment,
promptText: initialPromptText,
activeModelName: "gemini-3.5-flash-lite",
historyItems: [],
isBusy: false,
@@ -375,15 +519,18 @@
sessionId: null,
models: [
{ name: "gpt-5.4-mini", chat: chatWithOpenAI, key: "OPENAI_API_KEY" },
{ name: "gemini-3.5-flash-lite", chat: chatWithGemini, key: "GEMINI_API_KEY" },
{ name: "claude-haiku-4-5-20251001", chat: chatWithClaude, key: "ANTHROPIC_API_KEY" },
{ name: "gpt-5.6-luna", chat: chatWithOpenAI, key: window.env.OPENAI_API_KEY },
{ name: "gemini-3.5-flash-lite", chat: chatWithGemini, key: window.env.GEMINI_API_KEY },
{ name: "claude-haiku-4-5-20251001", chat: chatWithClaude, key: window.env.ANTHROPIC_API_KEY },
],
async init() {
this.sessionId = +new Date();
this.historyItems = loadHistory();
this.historyItems = this.attachment ? [] : loadHistory();
setTimeout(() => this.$refs.promptInput.focus(), 0);
if (this.attachment && window.env.AUTO_SUBMIT === true) {
setTimeout(() => this.useSuggestedQuestion(this.promptText.trim() || defaultVideoSummaryPrompt, true), 0);
}
},
async onClickMessage(e) {
@@ -423,6 +570,12 @@
getExcerpt: (msgs) => msgs.find((m) => m.role === "user")?.content.slice(0, 30) || "Empty chat",
renderMarkdown,
useSuggestedQuestion(question, submit = false) {
if (this.isBusy) return;
this.promptText = question;
this.$nextTick(() => this.$refs.promptInput.focus());
if (submit) this.handleSubmit();
},
async handleSubmit() {
const text = this.promptText.trim();
if ((!text && !this.pendingImages.length) || this.isBusy) return;
@@ -441,11 +594,13 @@
this.scrollToBottom();
try {
const modelObj = this.models.find((m) => m.name === this.activeModelName);
const modelObj = this.models.find((m) => m.name === (this.attachment ? "gemini-3.5-flash-lite" : this.activeModelName));
await modelObj.chat({
model: this.activeModelName,
apiKey: window.env[modelObj.key],
model: modelObj.name,
apiKey: modelObj.key,
messages: this.messages,
attachment: this.attachment,
systemInstruction: getSystemInstruction(this.attachment),
signal: this.abortController.signal,
onContent: (chunk) => {
fullResponse += chunk;
@@ -469,7 +624,7 @@
} finally {
this.isBusy = false;
this.abortController = null;
updateSessionHistory(this.sessionId, this.messages);
if (!this.attachment) updateSessionHistory(this.sessionId, this.messages);
}
},
@@ -478,6 +633,7 @@
},
handlePaste(e) {
if (this.attachment) return;
const items = [...(e.clipboardData?.items || [])];
const imageItems = items.filter((item) => item.type.startsWith("image/"));
if (!imageItems.length) return;
@@ -492,12 +648,31 @@
});
},
trimTo(msg) {
if (this.isBusy) return;
const idx = this.messages.indexOf(msg);
if (idx === -1) return;
this.messages = this.messages.slice(0, idx);
if (!this.attachment) updateSessionHistory(this.sessionId, this.messages);
this.promptText = msg.content;
this.pendingImages = msg.images ? JSON.parse(JSON.stringify(msg.images)) : [];
this.$nextTick(() => this.$refs.promptInput.focus());
},
restoreConversation(msgs) {
if (this.attachment) return;
this.messages = JSON.parse(JSON.stringify(msgs));
this.userHasScrolledUp = false;
this.$refs.historyPanel.removeAttribute("open");
this.scrollToBottom();
},
clearHistory() {
if (this.attachment) return;
if (!confirm("Clear all saved conversations?")) return;
this.historyItems = clearAllHistory();
this.$refs.historyPanel.removeAttribute("open");
},
});
function attachCopyButtons($el) {
@@ -519,15 +694,28 @@
return $placeholder.innerHTML;
}
async function chatWithOpenAI({ baseUrl = "https://api.openai.com/v1", onContent, onError, messages, signal, apiKey, model, extraHeaders = {}, ...rest }) {
async function chatWithOpenAI({ baseUrl = "https://api.openai.com/v1", onContent, onError, messages, signal, apiKey, model, extraHeaders = {}, systemInstruction, attachment, ...rest }) {
const lastUserIndex = messages.map((message) => message.role).lastIndexOf("user");
const apiMessages = messages.map(({ role, content, images }, index) => {
const textPart = {
type: "text",
text: content,
...(index === lastUserIndex ? { prompt_cache_breakpoint: { mode: "explicit" } } : {}),
};
const parts = [textPart, ...(images || []).map((img) => ({ type: "image_url", image_url: { url: img.dataUrl } }))];
return role === "system" || images?.length || index === lastUserIndex ? { role, content: parts } : { role, content };
});
const response = await fetch(`${baseUrl}/chat/completions`, {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: `Bearer ${apiKey}`, ...extraHeaders },
body: JSON.stringify({ model, messages: messages.map(({ role, content, images }) =>
images?.length
? { role, content: [{ type: "text", text: content }, ...images.map((img) => ({ type: "image_url", image_url: { url: img.dataUrl } }))] }
: { role, content }
), stream: true, ...rest }),
body: JSON.stringify({
model,
messages: apiMessages,
prompt_cache_key: "chat:index:v1",
prompt_cache_options: { mode: "explicit" },
stream: true,
...rest,
}),
signal,
});
if (!response.ok) {
@@ -556,26 +744,83 @@
}
async function chatWithClaude(args) {
return chatWithOpenAI({
...args,
baseUrl: "https://api.anthropic.com/v1",
extraHeaders: {
"x-api-key": args.apiKey,
const { onContent, onError, messages, signal, apiKey, model } = args;
const system = messages
.filter((message) => message.role === "system")
.map((message) => ({ type: "text", text: message.content }))
.filter((part) => part.text);
const conversation = messages
.filter((message) => message.role !== "system")
.map((message) => {
const images = (message.images || []).map((image) => ({
type: "image",
source: { type: "base64", media_type: image.mimeType, data: image.base64 },
}));
return {
role: message.role,
content: images.length ? [{ type: "text", text: message.content }, ...images] : message.content,
};
});
const response = await fetch("https://api.anthropic.com/v1/messages", {
method: "POST",
headers: {
"Content-Type": "application/json",
"x-api-key": apiKey,
"anthropic-version": "2023-06-01",
"anthropic-dangerous-direct-browser-access": "true",
},
apiKey: window.env.ANTHROPIC_API_KEY,
body: JSON.stringify({
model,
max_tokens: 4096,
system,
messages: conversation,
cache_control: { type: "ephemeral" },
stream: true,
}),
signal,
});
if (!response.ok) {
const text = await response.text();
await onError?.(new Error(`Anthropic API Error ${response.status}: ${text.trim()}`));
return;
}
async function chatWithGemini({ onContent, onError, messages, signal, apiKey = window.env.GEMINI_API_KEY, model, ...rest }) {
const contents = messages.map((msg, i, arr) => {
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const events = buffer.split("\n\n");
buffer = events.pop();
for (const event of events) {
const data = event
.split("\n")
.find((line) => line.startsWith("data:"))
?.slice(5)
.trim();
if (!data) continue;
try {
const parsed = JSON.parse(data);
if (parsed.type === "content_block_delta" && parsed.delta?.type === "text_delta") await onContent(parsed.delta.text);
} catch (error) {}
}
}
}
async function chatWithGemini({ onContent, onError, messages, signal, apiKey, model, systemInstruction, attachment, ...rest }) {
const conversation = messages.filter((msg) => msg.role !== "system");
const cachedContent = await getGeminiAttachmentCache({ model, apiKey, systemInstruction, attachment });
const contents = conversation.map((msg, i) => {
const role = msg.role === "assistant" ? "model" : "user";
let parts = [{ text: msg.content }];
// Add YouTube URLs as file data for the final user message
const isLastMessage = i === arr.length - 1;
if (isLastMessage && role === "user") {
// Preserve the existing ability to send a YouTube URL in a normal chat message.
const isLastMessage = i === conversation.length - 1;
if (!attachment && isLastMessage && role === "user") {
const youtubeUrlPattern = /https?:\/\/(www\.)?youtube\.com\/watch\?v=[\w-]+|https?:\/\/youtu\.be\/[\w-]+/g;
const youtubeUrls = msg.content.match(youtubeUrlPattern) || [];
const fileParts = youtubeUrls.map((url) => ({ file_data: { file_uri: url } }));
@@ -598,6 +843,7 @@
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
...(cachedContent ? { cachedContent } : systemInstruction ? { system_instruction: { parts: [{ text: systemInstruction }] } } : {}),
contents,
generationConfig: rest,
}),
@@ -740,6 +986,11 @@
return all[today];
};
const clearAllHistory = () => {
localStorage.removeItem(STORAGE_KEY);
return [];
};
document.addEventListener("alpine:init", () => {
Alpine.data("chatApp", chatApp);
});
+46 -62
View File
@@ -1,9 +1,8 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = ["httpx", "pillow"]
# dependencies = ["httpx", "pillow", "click"]
# ///
import argparse
import dataclasses
import logging
import os
@@ -16,6 +15,7 @@ from functools import partial
from pathlib import Path
from typing import Callable
import click
import httpx
from PIL import Image
@@ -326,11 +326,24 @@ def is_image(f: Path) -> bool:
# ---------------------------------------------------------------------------
# Subcommands
# CLI
# ---------------------------------------------------------------------------
def cmd_gallery(urls: list[str], name: str | None, cwd: Path | None = None) -> None:
_DEFAULT_CWD = Path("~/Downloads/temp/_imagesets").expanduser()
@click.group()
def cli() -> None:
pass
@cli.command("gallery")
@click.argument("urls", nargs=-1, required=True)
@click.option("--name", default=None)
@click.option("--cwd", type=Path, default=None)
def gallery_cmd(urls: tuple[str, ...], name: str | None, cwd: Path | None) -> None:
urls = list(urls)
resolved_name = name or _name_from_url(urls[0])
output_dir = (cwd or Path(".")) / resolved_name
@@ -346,9 +359,12 @@ def cmd_gallery(urls: list[str], name: str | None, cwd: Path | None = None) -> N
_extract_and_process(output_dir)
def cmd_filehost(
urls: list[str], name: str | None, token: str, cwd: Path | None = None
) -> None:
@cli.command("filehost")
@click.argument("urls", nargs=-1, required=True)
@click.option("--name", default=None)
@click.option("--cwd", type=Path, default=_DEFAULT_CWD)
@click.option("--token", default=os.getenv("ALLDEBRID_TOKEN"))
def filehost_cmd(urls: tuple[str, ...], name: str | None, cwd: Path, token: str) -> None:
resolved_name = name or _name_from_url(urls[0])
output_dir = (cwd or Path(".")) / resolved_name
output_dir.mkdir(parents=True, exist_ok=True)
@@ -376,7 +392,26 @@ def cmd_filehost(
_extract_and_process(output_dir)
def cmd_process(dirs: list[str]) -> None:
@cli.command("download")
@click.argument("urls", nargs=-1, required=True)
@click.option("--name", default=None)
@click.option("--cwd", type=Path, default=_DEFAULT_CWD)
@click.option("--token", default=os.getenv("ALLDEBRID_TOKEN"))
@click.pass_context
def download_cmd(
ctx: click.Context, urls: tuple[str, ...], name: str | None, cwd: Path, token: str
) -> None:
filehost_urls = [u for u in urls if "rg.to" in u or "rapidgator.net" in u]
gallery_urls = [u for u in urls if u not in set(filehost_urls)]
if filehost_urls:
ctx.invoke(filehost_cmd, urls=tuple(filehost_urls), name=name, cwd=cwd, token=token)
if gallery_urls:
ctx.invoke(gallery_cmd, urls=tuple(gallery_urls), name=name, cwd=cwd)
@cli.command("process")
@click.argument("dirs", nargs=-1, required=True)
def process_cmd(dirs: tuple[str, ...]) -> None:
for d in dirs:
p = Path(d)
if not p.exists():
@@ -400,63 +435,12 @@ def cmd_process(dirs: list[str]) -> None:
if originals:
print(f"trashing {len(originals)} original(s)")
move_to_trash(originals)
remove_empty_dirs(p)
else:
print(f"skipping {p}: already within size limits")
def cmd_download(
urls: list[str], name: str | None, token: str, cwd: Path | None = None
) -> None:
filehost_urls = [u for u in urls if "rg.to" in u or "rapidgator.net" in u]
gallery_urls = [u for u in urls if u not in set(filehost_urls)]
if filehost_urls:
cmd_filehost(filehost_urls, name, token, cwd)
if gallery_urls:
cmd_gallery(gallery_urls, name, cwd)
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def main() -> None:
print(os.getenv("PATH"))
parser = argparse.ArgumentParser(prog="download_imageset.py")
sub = parser.add_subparsers(dest="command", required=True)
gallery_p = sub.add_parser("gallery")
gallery_p.add_argument("urls", nargs="+")
gallery_p.add_argument("--name", default=None)
gallery_p.add_argument("--cwd", type=Path, default=None)
for cmd in ("filehost", "download"):
p = sub.add_parser(cmd)
p.add_argument("urls", nargs="+")
p.add_argument("--name", default=None)
p.add_argument(
"--cwd",
default=Path("~/Downloads/_temp/_imagesets").expanduser(),
type=Path,
)
p.add_argument("--token", default=os.getenv("ALLDEBRID_TOKEN"))
proc = sub.add_parser("process")
proc.add_argument("dirs", nargs="+")
args = parser.parse_args()
match args.command:
case "gallery":
cmd_gallery(args.urls, args.name, args.cwd)
case "filehost":
cmd_filehost(args.urls, args.name, args.token, cwd=args.cwd)
case "process":
cmd_process(args.dirs)
case "download":
cmd_download(args.urls, args.name, args.token, cwd=args.cwd)
rename_children(p, flatten=True)
remove_empty_dirs(p)
if __name__ == "__main__":
main()
cli()
Executable
+170
View File
@@ -0,0 +1,170 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = ["psycopg[binary]"]
# ///
"""
export_pg.py — run an arbitrary Postgres query via psycopg and stream the
results into a SQLite database in batches, without pulling the whole result
set into memory.
This uses a server-side (named) cursor: the query is planned and executed
by Postgres exactly once, and rows are then fetched from that single result
set in --page-size batches. This is the same thing a GUI SQL client does
when you "load more rows" — it is NOT the same as re-running the query with
a different LIMIT/OFFSET each time, which would redo the scan/filter/sort
from scratch on every page and be dramatically slower on a big query.
Example (the motivating case):
./export_pg.py \\
--dsn "service=mydb" \\
--query 'SELECT id, item_identifier FROM order_item_details' \\
--table order_item_details \\
-o export.sqlite3
Connection: --dsn is passed straight to psycopg.connect() as a conninfo
string (e.g. "host=... dbname=... user=..." or a postgresql:// URL). If
omitted, psycopg falls back to the standard PG* environment variables /
libpq defaults.
"""
import argparse
import datetime
import decimal
import json
import sqlite3
import sys
import time
import uuid
import psycopg
def sqlite_type_for(value) -> str:
if isinstance(value, bool) or isinstance(value, int):
return "INTEGER"
if isinstance(value, float):
return "REAL"
if isinstance(value, (bytes, bytearray, memoryview)):
return "BLOB"
return "TEXT"
def infer_sqlite_type(values: list) -> str:
for v in values:
if v is not None:
return sqlite_type_for(v)
return "TEXT"
def adapt(value):
if value is None or isinstance(value, (int, float, str, bytes)):
return value
if isinstance(value, memoryview):
return bytes(value)
if isinstance(value, decimal.Decimal):
return str(value)
if isinstance(value, (datetime.date, datetime.time, datetime.datetime)):
return value.isoformat()
if isinstance(value, uuid.UUID):
return str(value)
if isinstance(value, (list, dict)):
return json.dumps(value)
return str(value)
def main():
class Formatter(argparse.RawDescriptionHelpFormatter,
argparse.ArgumentDefaultsHelpFormatter):
pass
ap = argparse.ArgumentParser(description=__doc__, formatter_class=Formatter)
ap.add_argument("--query", required=True,
help="query to run (a trailing ';' is stripped automatically)")
ap.add_argument("-o", "--output", required=True, help="path to SQLite db file")
ap.add_argument("--table", required=True, help="destination table name")
ap.add_argument("--dsn", help="psycopg conninfo string / postgresql:// URL")
ap.add_argument("--page-size", type=int, default=2_000,
help="rows fetched from the server-side cursor per batch")
ap.add_argument("--if-exists", choices=["fail", "replace", "append"],
default="fail", help="behavior when --table already exists")
args = ap.parse_args()
query = args.query.strip().rstrip(";").strip()
con = sqlite3.connect(args.output)
cur = con.cursor()
exists = cur.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name=?",
(args.table,),
).fetchone()
if exists:
if args.if_exists == "fail":
print(f"table {args.table!r} already exists in {args.output} "
"(use --if-exists replace|append)", file=sys.stderr)
sys.exit(1)
elif args.if_exists == "replace":
cur.execute(f'DROP TABLE "{args.table}"')
exists = None
pg = psycopg.connect(args.dsn) if args.dsn else psycopg.connect()
total = 0
columns = None
col_types = None
start = time.monotonic()
show_progress = sys.stderr.isatty()
def report(done: bool):
elapsed = time.monotonic() - start
rate = total / elapsed if elapsed > 0 else 0
msg = f"{total} rows written ({rate:,.0f} rows/s, {elapsed:.0f}s elapsed)"
if show_progress and not done:
print(f"\r{msg}", end="", file=sys.stderr, flush=True)
else:
print(msg, file=sys.stderr)
try:
print(query, file=sys.stderr)
with pg.cursor(name="export_pg") as pg_cur:
pg_cur.itersize = args.page_size
pg_cur.execute(query)
while True:
rows = pg_cur.fetchmany(args.page_size)
if not rows:
break
if columns is None:
columns = [d.name for d in pg_cur.description]
col_types = [
infer_sqlite_type([r[i] for r in rows])
for i in range(len(columns))
]
if not exists:
cols_sql = ", ".join(
f'"{c}" {t}' for c, t in zip(columns, col_types)
)
cur.execute(f'CREATE TABLE "{args.table}" ({cols_sql})')
placeholders = ", ".join("?" for _ in columns)
converted = [[adapt(v) for v in row] for row in rows]
cur.executemany(
f'INSERT INTO "{args.table}" VALUES ({placeholders})', converted
)
con.commit()
total += len(rows)
report(done=len(rows) < args.page_size)
finally:
pg.close()
con.close()
print(f"done: {total} rows written to {args.output}:{args.table}",
file=sys.stderr)
if __name__ == "__main__":
main()
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python3.10
#!/usr/bin/env python3
import argparse
import dataclasses
import datetime
Executable
+30
View File
@@ -0,0 +1,30 @@
#!/bin/bash
set -euo pipefail
MPV=/opt/homebrew/bin/mpv
if [[ $# -eq 0 ]]; then
echo "Usage: $(basename "$0") <file|url> [file|url ...]" >&2
exit 1
fi
IFS=$'\n' read -d '' -r -a TARGETS < <(python3 -c '
import sys, urllib.parse as u
for arg in sys.argv[1:]:
for line in arg.splitlines():
line = line.strip()
if not line:
continue
if line.startswith("mpv://"):
try:
parsed = u.urlparse(line)
line = u.parse_qs(parsed.query)["url"][0]
except (KeyError, IndexError):
pass
print(line)
' "$@") || true
[[ ${#TARGETS[@]} -eq 0 ]] && exit 1
"$MPV" --ontop --on-all-workspaces --volume=0 "${TARGETS[@]}" &
+9 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python3.10
#!/usr/bin/env python3
import argparse
import dataclasses
@@ -56,6 +56,14 @@ def rename_children(source_dir: Path, flatten: bool) -> None:
write_undo_script(Path.cwd(), jobs)
remove_empty_dirs(source_dir)
def remove_empty_dirs(root: Path) -> None:
for path in sorted(root.rglob("*"), reverse=True):
if path.is_dir() and not any(path.iterdir()):
path.rmdir()
@dataclasses.dataclass
class MoveOp:
+22 -4
View File
@@ -208,6 +208,7 @@
<div x-show="playlistLink">
<a class="button" :href="playlistLink">Play all with mpv</a>
<a class="button" :href="gridLink">Play all in a grid</a>
</div>
<div class="filter-bar gapped small">
@@ -236,7 +237,7 @@
},
sort: { by: null, dir: "asc" },
buildAlfredURL(url) {
return `alfred://runtrigger/piracy/direct_link/?argument=${encodeURIComponent(url)}`;
return `alfred://runtrigger/dev.abdus.alfred.media/video/?argument=${encodeURIComponent(url)}`;
},
toggleSort(field) {
if (this.sort.by === field) {
@@ -320,7 +321,7 @@
await this.search();
}
},
get playlistLink() {
get playableURLs() {
const playable = this.visibleResults.some((item) => item.file_path.startsWith("/mnt/box/files"));
if (!playable) {
return null;
@@ -329,9 +330,26 @@
.filter((item) => item.file_path.startsWith("/mnt/box/files"))
.map((item) => item.download_url)
.sort(() => Math.random() - 0.5);
const playlistArg = encodeURIComponent(urls.join("\n"));
return `alfred://runtrigger/dev.abdus.labs/mpv_playlist/?argument=${playlistArg}`;
return urls;
},
get playlistLink() {
const urls = this.playableURLs;
if (!urls) {
return null;
}
const playlistArg = encodeURIComponent(urls.join("\n"));
return `alfred://runtrigger/dev.abdus.alfred.media/mpv/?argument=${playlistArg}`;
},
get gridLink() {
const urls = this.playableURLs;
if (!urls) {
return null;
}
const playlistArg = encodeURIComponent(urls.join("\n"));
return `alfred://runtrigger/dev.abdus.alfred.media/gridvid/?argument=${playlistArg}`;
},
async search() {
const response = await fetch("/search", {
method: "POST",
+31 -10
View File
@@ -14,6 +14,30 @@ from PIL import Image
# --- Utility Functions ---
class ImageFormat(enum.Enum):
JPEG = "jpeg"
PNG = "png"
WEBP = "webp"
@property
def suffix(self) -> str:
return ".jpg" if self is self.JPEG else f".{self.value}"
def sniff_image_format(image_path: Path) -> ImageFormat:
"""Identify a supported image format from its file signature."""
with image_path.open("rb") as file:
header = file.read(12)
if header.startswith(b"\xff\xd8\xff"):
return ImageFormat.JPEG
if header.startswith(b"\x89PNG\r\n\x1a\n"):
return ImageFormat.PNG
if len(header) >= 12 and header[:4] == b"RIFF" and header[8:12] == b"WEBP":
return ImageFormat.WEBP
raise ValueError(f"Unsupported or unrecognized image format: {image_path}")
def read_dims_jpeg(image_path: Path) -> tuple[int, int]:
with image_path.open("rb") as file:
if file.read(2) != b"\xff\xd8":
@@ -129,14 +153,14 @@ def read_dims_webp(image_path: Path) -> tuple[int, int]:
def read_dims(image_path: Path) -> tuple[int, int]:
if image_path.suffix.lower() in [".jpg", ".jpeg"]:
image_format = sniff_image_format(image_path)
if image_format is ImageFormat.JPEG:
return read_dims_jpeg(image_path)
elif image_path.suffix.lower() == ".png":
elif image_format is ImageFormat.PNG:
return read_dims_png(image_path)
elif image_path.suffix.lower() == ".webp":
elif image_format is ImageFormat.WEBP:
return read_dims_webp(image_path)
else:
raise ValueError(f"Unsupported image format: {image_path.suffix}")
raise AssertionError(f"Unhandled image format: {image_format}")
def parse_args() -> argparse.Namespace:
@@ -188,11 +212,8 @@ def main():
logging.warning(f"{image_path} is not a file. Skipping.")
continue
if image_path.suffix.lower() not in [".jpg", ".jpeg", ".png", ".webp"]:
logging.warning(f"{image_path} is not a JPEG, PNG, or WebP file. Skipping.")
continue
try:
image_format = sniff_image_format(image_path)
width, height = read_dims(image_path)
target_dir = None
@@ -208,7 +229,7 @@ def main():
make_dir(target_dir)
target_path = target_dir / image_path.name
target_path = target_path.with_suffix(".jpg")
target_path = target_path.with_suffix(image_format.suffix)
image_path.rename(target_path)
logging.info(f"Moved {image_path} to {target_path}")
Executable
+472
View File
@@ -0,0 +1,472 @@
#!/usr/bin/env python3
"""Create and restore a marked ZIP backup of Alfred preferences over WebDAV."""
from __future__ import annotations
import argparse
import base64
import json
import logging
import os
import shutil
import socket
import stat
import tempfile
import time
import urllib.error
import urllib.parse
import urllib.request
import zipfile
from datetime import datetime, timezone
from pathlib import Path, PurePosixPath
PACKAGE_NAME = "Alfred.alfredpreferences"
ARCHIVE_NAME = f"{PACKAGE_NAME}.sync-backup.zip"
MARKER_NAME = ".alfred-sync-backup"
FORMAT_VERSION = 1
DEFAULT_WEBDAV_URL = (
"https://u201686:T6672ICVoWAedECH@u201686.your-storagebox.de/"
)
DEFAULT_REMOTE_PATH = "backup/macs/alfred"
WEBDAV_TIMEOUT = 30
LOGGER = logging.getLogger("sync-alfred")
class SyncError(RuntimeError):
pass
class ProgressIO:
"""Wrap a readable binary file and report transfer progress once per second."""
def __init__(
self,
file_obj,
*,
total: int | None = None,
label: str = "Progress",
) -> None:
self.file_obj = file_obj
self.total = total
self.label = label
self.transferred = 0
self.started_at = time.monotonic()
self.last_report_at = self.started_at - 1
self.finished = False
self._report(force=True)
def __getattr__(self, name):
return getattr(self.file_obj, name)
def __len__(self) -> int:
return self.total or 0
def read(self, size: int = -1):
chunk = self.file_obj.read(size)
if chunk:
self.transferred += len(chunk)
self._report()
else:
self._report(force=True, complete=True)
return chunk
def finish(self) -> None:
self._report(force=True, complete=True)
@staticmethod
def _format_bytes(value: float) -> str:
units = ("B", "KiB", "MiB", "GiB")
for unit in units:
if value < 1024 or unit == units[-1]:
return f"{value:.1f} {unit}"
value /= 1024
return f"{value:.1f} GiB"
def _report(self, *, force: bool = False, complete: bool = False) -> None:
if self.finished:
return
now = time.monotonic()
if not force and now - self.last_report_at < 1:
return
self.last_report_at = now
elapsed = max(now - self.started_at, 0.001)
transferred = self._format_bytes(self.transferred)
rate = self._format_bytes(self.transferred / elapsed)
if self.total is None:
message = f"{self.label}: {transferred} at {rate}/s"
else:
percent = 100 * self.transferred / self.total if self.total else 100
total = self._format_bytes(self.total)
message = f"{self.label}: {transferred} / {total} ({percent:.1f}%) at {rate}/s"
LOGGER.info(message)
if complete:
self.finished = True
class WebDAVClient:
def __init__(self, base_url: str, remote_path: str) -> None:
parsed = urllib.parse.urlsplit(base_url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise SyncError("invalid WebDAV URL")
if parsed.query or parsed.fragment:
raise SyncError("WebDAV URL must not contain a query or fragment")
username = urllib.parse.unquote(parsed.username or "")
password = urllib.parse.unquote(parsed.password or "")
if not username:
username = os.environ.get("ALFRED_WEBDAV_USER", "")
if not password:
password = os.environ.get("ALFRED_WEBDAV_PASSWORD", "")
# Strip credentials from request URLs and diagnostic output.
netloc = parsed.netloc.rsplit("@", 1)[-1]
base_path = parsed.path.rstrip("/")
self.base_url = urllib.parse.urlunsplit(
(parsed.scheme, netloc, f"{base_path}/", "", "")
)
self.remote_parts = self._parse_remote_path(remote_path)
self.auth_header = None
password_manager = urllib.request.HTTPPasswordMgrWithDefaultRealm()
if username and password:
encoded_credentials = base64.b64encode(
f"{username}:{password}".encode("utf-8")
).decode("ascii")
self.auth_header = f"Basic {encoded_credentials}"
origin = urllib.parse.urlunsplit(
(parsed.scheme, netloc, "/", "", "")
)
password_manager.add_password(None, origin, username, password)
self.opener = urllib.request.build_opener(
urllib.request.HTTPBasicAuthHandler(password_manager)
)
@staticmethod
def _parse_remote_path(remote_path: str) -> tuple[str, ...]:
parts = tuple(remote_path.strip("/").split("/"))
if not parts or any(not part or part in (".", "..") for part in parts):
raise SyncError(f"invalid WebDAV remote path: {remote_path!r}")
return parts
def _url_for(self, parts: tuple[str, ...]) -> str:
encoded = "/".join(urllib.parse.quote(part, safe="") for part in parts)
return f"{self.base_url.rstrip('/')}/{encoded}"
@property
def archive_url(self) -> str:
return self._url_for(self.remote_parts + (ARCHIVE_NAME,))
@property
def display_location(self) -> str:
return self.archive_url
def _request(
self,
method: str,
url: str,
*,
data: object | None = None,
headers: dict[str, str] | None = None,
) -> urllib.response.addinfourl:
request_headers = dict(headers or {})
if self.auth_header:
request_headers.setdefault("Authorization", self.auth_header)
request = urllib.request.Request(
url, data=data, headers=request_headers, method=method
)
try:
return self.opener.open(request, timeout=WEBDAV_TIMEOUT)
except urllib.error.HTTPError as exc:
detail = f"HTTP {exc.code} {exc.reason}"
raise SyncError(
f"WebDAV {method} failed for remote archive: {detail}"
) from exc
except urllib.error.URLError as exc:
raise SyncError(f"WebDAV {method} failed: {exc.reason}") from exc
def ensure_remote_directory(self) -> None:
for index in range(1, len(self.remote_parts) + 1):
# WebDAV collections are directory-like resources and this server
# redirects collection URLs without their trailing slash.
url = f"{self._url_for(self.remote_parts[:index])}/"
headers = {"Authorization": self.auth_header} if self.auth_header else {}
request = urllib.request.Request(url, headers=headers, method="MKCOL")
try:
response = self.opener.open(request, timeout=WEBDAV_TIMEOUT)
except urllib.error.HTTPError as exc:
# WebDAV uses 405 when MKCOL is sent for an existing collection.
if exc.code != 405:
detail = f"HTTP {exc.code} {exc.reason}"
raise SyncError(
f"WebDAV MKCOL failed for remote directory: {detail}"
) from exc
exc.close()
except urllib.error.URLError as exc:
raise SyncError(f"WebDAV MKCOL failed: {exc.reason}") from exc
else:
response.close()
def upload(self, archive_path: Path) -> None:
self.ensure_remote_directory()
try:
total = archive_path.stat().st_size
source = archive_path.open("rb")
except OSError as exc:
raise SyncError(f"could not open archive {archive_path}: {exc}") from exc
progress = ProgressIO(source, total=total, label="Uploading")
try:
with source:
with self._request(
"PUT",
self.archive_url,
data=progress,
headers={
"Content-Type": "application/zip",
"Content-Length": str(total),
},
) as response:
response.read()
finally:
progress.finish()
def download(self, archive_path: Path) -> None:
with self._request("GET", self.archive_url) as response:
try:
with archive_path.open("wb") as output:
shutil.copyfileobj(response, output)
output.flush()
os.fsync(output.fileno())
except OSError as exc:
raise SyncError(
f"could not write archive {archive_path}: {exc}"
) from exc
def configured_paths() -> tuple[Path, WebDAVClient]:
local_dir = Path(
os.environ.get("ALFRED_LOCAL_DIR", "~/Documents/alfred")
).expanduser()
webdav = WebDAVClient(
os.environ.get("ALFRED_WEBDAV_URL", DEFAULT_WEBDAV_URL),
os.environ.get("ALFRED_REMOTE_PATH", DEFAULT_REMOTE_PATH),
)
return local_dir, webdav
def marker(kind: str) -> bytes:
contents = {
"format": FORMAT_VERSION,
"package": PACKAGE_NAME,
"kind": kind,
"created_utc": datetime.now(timezone.utc).isoformat(),
"source_host": socket.gethostname(),
}
return (json.dumps(contents, sort_keys=True) + "\n").encode("utf-8")
def create_archive(source_dir: Path, archive_path: Path, kind: str) -> None:
if not source_dir.is_dir():
raise SyncError(f"local preferences directory not found: {source_dir}")
try:
with zipfile.ZipFile(
archive_path, "w", compression=zipfile.ZIP_DEFLATED
) as archive:
archive.write(source_dir, PACKAGE_NAME)
for path in sorted(source_dir.rglob("*"), key=lambda item: item.as_posix()):
if path.is_symlink():
raise SyncError(f"symlinks are not supported in preferences: {path}")
relative = path.relative_to(source_dir)
archive.write(
path,
PurePosixPath(PACKAGE_NAME, *relative.parts).as_posix(),
)
archive.writestr(MARKER_NAME, marker(kind))
except OSError as exc:
raise SyncError(f"could not create archive {archive_path}: {exc}") from exc
def validate_archive(archive_path: Path) -> None:
try:
with zipfile.ZipFile(archive_path) as archive:
broken_file = archive.testzip()
if broken_file is not None:
raise SyncError(f"backup archive is corrupt: {broken_file}")
names = archive.namelist()
if names.count(MARKER_NAME) != 1:
raise SyncError(f"backup archive must contain one {MARKER_NAME} marker")
try:
metadata = json.loads(archive.read(MARKER_NAME).decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError, KeyError) as exc:
raise SyncError("backup marker is invalid") from exc
if metadata.get("format") != FORMAT_VERSION:
raise SyncError("unsupported Alfred backup format")
if metadata.get("package") != PACKAGE_NAME:
raise SyncError("backup marker is for a different package")
if metadata.get("kind") != "backup":
raise SyncError("archive is not a primary Alfred backup")
seen: set[str] = set()
has_package = False
for name in names:
if name in seen:
raise SyncError(f"duplicate file in backup archive: {name}")
seen.add(name)
if "\\" in name:
raise SyncError(f"unsafe path in backup archive: {name}")
path = PurePosixPath(name)
if path.is_absolute() or ".." in path.parts:
raise SyncError(f"unsafe path in backup archive: {name}")
if name == MARKER_NAME:
continue
if name == PACKAGE_NAME or name.startswith(f"{PACKAGE_NAME}/"):
has_package = True
continue
raise SyncError(f"unexpected file in backup archive: {name}")
if not has_package:
raise SyncError(f"backup archive does not contain {PACKAGE_NAME}")
except zipfile.BadZipFile as exc:
raise SyncError(f"backup archive is not a valid ZIP: {archive_path}") from exc
def extract_archive(archive_path: Path, destination: Path) -> Path:
package_destination = destination / PACKAGE_NAME
destination.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(archive_path) as archive:
for info in archive.infolist():
if info.filename == MARKER_NAME:
continue
relative = PurePosixPath(info.filename)
target = destination.joinpath(*relative.parts)
if info.is_dir() or info.filename.endswith("/"):
target.mkdir(parents=True, exist_ok=True)
continue
mode = (info.external_attr >> 16) & 0xFFFF
if stat.S_ISLNK(mode):
raise SyncError(f"symlinks are not supported in backup archives: {info.filename}")
target.parent.mkdir(parents=True, exist_ok=True)
with archive.open(info) as source, target.open("wb") as output:
shutil.copyfileobj(source, output)
permissions = mode & 0o777
if permissions:
target.chmod(permissions)
if not package_destination.is_dir():
raise SyncError(f"backup archive does not contain {PACKAGE_NAME}")
return package_destination
def unique_backup_path(local_dir: Path) -> Path:
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
candidate = local_dir / f"{PACKAGE_NAME}.before-restore-{timestamp}.zip"
if not candidate.exists():
return candidate
return local_dir / f"{PACKAGE_NAME}.before-restore-{timestamp}-{os.getpid()}.zip"
def backup_current_preferences(local_dir: Path, temporary_archive: Path) -> Path | None:
package = local_dir / PACKAGE_NAME
if not package.exists():
return None
if not package.is_dir():
raise SyncError(f"local preferences path is not a directory: {package}")
backup_path = unique_backup_path(local_dir)
create_archive(package, temporary_archive, "pre-restore")
os.replace(temporary_archive, backup_path)
return backup_path
def backup(local_dir: Path, webdav: WebDAVClient) -> None:
local_dir.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix=".alfred-backup-", dir=local_dir) as temp:
archive = Path(temp) / ARCHIVE_NAME
create_archive(local_dir / PACKAGE_NAME, archive, "backup")
webdav.upload(archive)
LOGGER.info("Backed up %s to %s", PACKAGE_NAME, webdav.display_location)
def restore(local_dir: Path, webdav: WebDAVClient) -> None:
local_dir.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix=".alfred-restore-", dir=local_dir) as temp:
temp_dir = Path(temp)
archive = temp_dir / ARCHIVE_NAME
webdav.download(archive)
validate_archive(archive)
restored_package = extract_archive(archive, temp_dir)
previous_backup = backup_current_preferences(
local_dir, temp_dir / "current-pre-restore.zip"
)
if previous_backup is not None:
LOGGER.info("Backed up current preferences to %s", previous_backup)
old_container = Path(
tempfile.mkdtemp(prefix=".alfred-old-", dir=local_dir)
)
old_package = old_container / PACKAGE_NAME
current_package = local_dir / PACKAGE_NAME
try:
if current_package.exists():
os.replace(current_package, old_package)
os.replace(restored_package, current_package)
except OSError as exc:
if old_package.exists() and not current_package.exists():
os.replace(old_package, current_package)
raise SyncError(f"could not install restored preferences: {exc}") from exc
else:
shutil.rmtree(old_container)
LOGGER.info("Restored %s from %s", PACKAGE_NAME, webdav.display_location)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Back up or restore Alfred preferences using a marked ZIP archive."
)
parser.add_argument(
"mode",
nargs="?",
default="backup",
choices=("backup", "push", "restore", "pull", "apply"),
help="backup (default) or restore; push/pull/apply are aliases",
)
return parser.parse_args()
def main() -> int:
args = parse_args()
logging.basicConfig(level=logging.INFO, format="%(message)s")
try:
local_dir, webdav = configured_paths()
if args.mode in ("backup", "push"):
backup(local_dir, webdav)
else:
restore(local_dir, webdav)
except SyncError as exc:
LOGGER.error("sync-alfred: %s", exc)
return 1
except OSError as exc:
LOGGER.error("sync-alfred: filesystem error: %s", exc)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
+56 -1
View File
@@ -57,6 +57,21 @@ def transcript_to_srt(transcript: list[dict]) -> str:
return "\n".join(srt_lines)
PRIORITY_LANGUAGES = ("de", "en", "tr")
def pick_auto_language(transcript_list) -> str:
available = {t.language_code: t for t in transcript_list}
if not available:
raise ValueError("No transcripts available for this video")
for lang in PRIORITY_LANGUAGES:
if lang in available:
return lang
return next(iter(available))
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Fetch YouTube video transcripts",
@@ -69,6 +84,18 @@ def parse_args() -> argparse.Namespace:
metavar="FILE_PATH",
help="Save transcript as SRT file to the specified path",
)
lang_group = parser.add_mutually_exclusive_group()
lang_group.add_argument(
"--language",
metavar="LANG",
help="Fetch transcript in the given language code (exits with error if unavailable)",
)
lang_group.add_argument(
"--auto",
action="store_true",
help=f"Auto-pick a language, preferring {', '.join(PRIORITY_LANGUAGES)} in that order, "
"then falling back to the first available generated/translated language",
)
return parser.parse_args()
@@ -80,14 +107,42 @@ def main() -> None:
logging.info(f"Fetching transcript for video ID: {video_id}")
ytt_api = YouTubeTranscriptApi()
if args.language:
transcript_list = ytt_api.list(video_id)
try:
chosen = transcript_list.find_transcript([args.language])
except Exception:
available = ", ".join(sorted(t.language_code for t in transcript_list))
logging.error(
f"Language '{args.language}' not available. Available languages: {available}"
)
sys.exit(1)
logging.info(f"Using language: {chosen.language_code}")
transcript = chosen.fetch()
elif args.auto:
transcript_list = ytt_api.list(video_id)
lang = pick_auto_language(transcript_list)
logging.info(f"Auto-selected language: {lang}")
transcript = transcript_list.find_transcript([lang]).fetch()
else:
try:
transcript = ytt_api.fetch(video_id)
except Exception:
transcript_list = ytt_api.list(video_id)
available = ", ".join(sorted(t.language_code for t in transcript_list))
logging.error(
f"No transcript available in the default language. "
f"Available languages: {available}. Use --language or --auto."
)
sys.exit(1)
if args.save_srt:
srt_content = transcript_to_srt(transcript.to_raw_data())
args.save_srt.write_text(srt_content)
logging.info(f"SRT file saved to: {args.save_srt}")
else:
text = "\n".join(snippet.text for snippet in transcript)
text = " ".join(snippet.text for snippet in transcript)
print(text)
except Exception as e: