结构调整
This commit is contained in:
@@ -0,0 +1,448 @@
|
||||
---
|
||||
name: "calibre-cleaner"
|
||||
description: "清洗 Calibre 书库:修复损坏书名/提取作者/检测编码损坏、小文件的批量删除"
|
||||
---
|
||||
|
||||
# Calibre 书库清洗 Skill
|
||||
|
||||
## 概述
|
||||
|
||||
Calibre 书库整理三件套:
|
||||
|
||||
1. **书名清洗** —— 修复从 txt 导入产生的 HTML 标签/长文本/乱码书名,截取前 20 个中文字符,移除网站链接
|
||||
2. **作者提取** —— 从书名或原始文件名 `data.name` 中自动提取作者名(匹配 `作者:xxx`、`-xxx`、`BY xxx` 等格式)
|
||||
3. **脏数据清理** —— 检测并删除内容损坏的 txt(中文 < 10%)、小于 10KB 的垃圾文件、Python 列表格式垃圾标题
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 书库路径:`/home/yangxuan/calibre/books`
|
||||
- 元数据文件:`/home/yangxuan/calibre/books/metadata.db`
|
||||
- 运行前须停止其他 Calibre 进程(桌面版、Docker Calibre-Web)
|
||||
- 运行前 **必须备份** `metadata.db`
|
||||
|
||||
---
|
||||
|
||||
## 一、书名清洗 & 作者提取
|
||||
|
||||
### `shorten(s, max_bytes=120)`
|
||||
|
||||
将字符串截断到指定字节数,避免因 UTF-8 中文字符跨字节而损坏。
|
||||
|
||||
```python
|
||||
def shorten(s, max_bytes=120):
|
||||
encoded = s.encode('utf-8')
|
||||
if len(encoded) <= max_bytes:
|
||||
return s
|
||||
try:
|
||||
return encoded[:max_bytes].decode('utf-8')
|
||||
except:
|
||||
return encoded[:max_bytes-3].decode('utf-8', errors='ignore')
|
||||
```
|
||||
|
||||
### `clean_title(title)`
|
||||
|
||||
```python
|
||||
def clean_title(title):
|
||||
t = title
|
||||
# 移除 HTML 标签
|
||||
t = re.sub(r'<[^>]+>', '', t)
|
||||
# 替换 _BR_ / _BR 标记
|
||||
t = re.sub(r'_BR_', ' ', t, flags=re.IGNORECASE)
|
||||
t = re.sub(r'_BR(?![a-zA-Z])', ' ', t)
|
||||
# 移除 _fontxxx_ / _imgxxx_ 标记
|
||||
t = re.sub(r'_font[^_]*_', '', t)
|
||||
t = re.sub(r'_img[^_]*_', '', t)
|
||||
# 移除网站链接
|
||||
t = re.sub(r'[\((]www\.[^))]+[\))]', '', t)
|
||||
t = re.sub(r'^[a-zA-Z0-9.-]+\.com@', '', t)
|
||||
t = re.sub(r'[-–—][a-zA-Z0-9.-]+\.com$', '', t)
|
||||
t = re.sub(r'【[^】]*?(?:下载|网站|书城|声明|提供)[^】]*】', '', t)
|
||||
t = re.sub(r'[::].*?(?:本文由|本电子书).*$', '', t)
|
||||
t = re.sub(r'[//]\s*www\.[^\s]+.*$', '', t)
|
||||
t = re.sub(r'[,,。]?https?://[^\s\))\]]+', '', t)
|
||||
# 移除声明文字当书名
|
||||
t = re.sub(r'^内容简介[】】]?', '', t)
|
||||
# 移除作者/翻译标记(批量清洗时用)
|
||||
t = re.sub(r'[ \s]*?(?:作者|翻译|原?作者)[::].*$', '', t, flags=re.DOTALL)
|
||||
# 移除 Unknown/未知 尾巴
|
||||
t = re.sub(r'\s*-\s*(Unknown|未知)\s*$', '', t)
|
||||
# 合并多余空格
|
||||
t = re.sub(r'\s+', ' ', t).strip()
|
||||
# 截取前 20 个中文字符(保留非中文直到第 20 个中文出现)
|
||||
chinese_count = 0
|
||||
pos = 0
|
||||
for i, c in enumerate(t):
|
||||
if '\u4e00' <= c <= '\u9fff':
|
||||
chinese_count += 1
|
||||
if chinese_count == 20:
|
||||
pos = i + 1
|
||||
break
|
||||
if chinese_count > 20:
|
||||
t = t[:pos]
|
||||
# 超长截短
|
||||
if len(t) > 100:
|
||||
m = re.search(r'[\u4e00-\u9fff]{2,}', t)
|
||||
if m:
|
||||
t = t[m.start():]
|
||||
t = t[:60].rstrip(',。;:、!?…\ufffd\'"').rstrip('.,;:!?\'').strip()
|
||||
# 移除控制字符
|
||||
t = re.sub(r'[\x00-\x08\x0b\x0c\x0e-\x1f]', '', t)
|
||||
return shorten(t, 100).strip() or 'Unknown'
|
||||
```
|
||||
|
||||
### 作者提取规则(从文件中同时匹配 title 和 data.name)
|
||||
|
||||
优先从 `data.name`(原始文件名)提取,更准确:
|
||||
|
||||
```python
|
||||
EXCLUDE_AUTHORS = {'正文', '内容简介', ''}
|
||||
|
||||
def extract_author_from_name(file_name):
|
||||
"""从 data.name(原始文件名)提取作者"""
|
||||
name = file_name
|
||||
# 去掉尾巴
|
||||
name = re.sub(r'\s*-\s*(Unknown|未知)\s*$', '', name)
|
||||
# 匹配作者:xxx
|
||||
m = re.search(r'作者[::]\s*([^\s_)\)《\[【\]】\-,]+)', name)
|
||||
if m:
|
||||
candidate = m.group(1).strip()
|
||||
if candidate not in EXCLUDE_AUTHORS and len(candidate) < 30:
|
||||
return candidate
|
||||
# 匹配翻译:xxx
|
||||
m = re.search(r'翻译[::]\s*([^\s_)\)《\[【\]】\-,]+)', name)
|
||||
if m:
|
||||
candidate = m.group(1).strip()
|
||||
if len(candidate) < 20:
|
||||
return candidate
|
||||
# 匹配原作者:xxx
|
||||
m = re.search(r'原?作者[::]\s*([^\s_)\)《\[【\]】\-,]+)', name)
|
||||
if m:
|
||||
candidate = m.group(1).strip()
|
||||
if len(candidate) < 30:
|
||||
return candidate
|
||||
return None
|
||||
|
||||
def extract_author_from_title(title):
|
||||
"""从 title 提取作者(降级方案,仅在 data.name 无用时)"""
|
||||
m = re.search(r'作者[::]\s*([^\s,,、\))《\[]+)', title)
|
||||
if m:
|
||||
name = m.group(1).strip()
|
||||
if name not in EXCLUDE_AUTHORS and not re.match(r'^第\d+[章节]', name):
|
||||
return name
|
||||
m = re.search(r'翻译[::]\s*([^\s,,、\))《\[]+)', title)
|
||||
if m:
|
||||
return m.group(1).strip()
|
||||
m = re.search(r'\b[Bb][Yy]\s+([^\s,,、\))《\[]+)', title)
|
||||
if m:
|
||||
name = m.group(1).strip()
|
||||
if name not in EXCLUDE_AUTHORS:
|
||||
return name
|
||||
m = re.search(r'-([^-()【】\[\]{}《》\s]{2,30})$', title)
|
||||
if m:
|
||||
name = m.group(1).strip()
|
||||
if name not in EXCLUDE_AUTHORS and not re.match(r'^\d+', name):
|
||||
return name
|
||||
m = re.search(r'&([^&()【】\[\]\s]{2,30})$', title)
|
||||
if m:
|
||||
name = m.group(1).strip()
|
||||
if name not in EXCLUDE_AUTHORS:
|
||||
return name
|
||||
return None
|
||||
```
|
||||
|
||||
### 坏书检测条件
|
||||
|
||||
```python
|
||||
is_bad = (
|
||||
len(title) > 100 or
|
||||
'<img' in title or '<BR' in title or '<font' in title or
|
||||
'\ufffd' in title or
|
||||
len(title.encode('utf-8')) > 150
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、脏数据清理
|
||||
|
||||
### 2.1 检测条件
|
||||
|
||||
扫描书库中所有 TXT 文件,读取前 20KB 检测中文比例:
|
||||
|
||||
- **内容损坏/二进制垃圾**:中文比例 < 10%(标记为待删除)
|
||||
- **小文件垃圾**:`uncompressed_size < 10240`(10KB 以下)
|
||||
- **垃圾格式标题**:Python 列表格式(含 `'xxx', 'R-18'` 单引号列表)、转义字符(`\n`、`\u3000`、`id=`)、HTML 实体(`raquo;`、`laquo;`)
|
||||
|
||||
```python
|
||||
def is_garbage_format_title(title):
|
||||
"""检测标题是否为垃圾格式"""
|
||||
import re
|
||||
if re.search(r"'.*',\s*'R-18'", title):
|
||||
return True
|
||||
if re.search(r'[\\]u[0-9a-fA-F]{4}', title):
|
||||
return True
|
||||
if re.search(r'id=|pid=', title):
|
||||
return True
|
||||
if 'raquo;' in title or 'laquo;' in title:
|
||||
return True
|
||||
if '【内容简介】' in title or '内容简介】' in title:
|
||||
return True
|
||||
return False
|
||||
|
||||
def is_dirty_content(full_path):
|
||||
"""检测文件内容是否为垃圾"""
|
||||
if not os.path.exists(full_path):
|
||||
return True # 文件丢失也算脏
|
||||
with open(full_path, 'rb') as f:
|
||||
raw = f.read(20480)
|
||||
text = raw.decode('utf-8', errors='replace')
|
||||
chinese = sum(1 for c in text if '\u4e00' <= c <= '\u9fff')
|
||||
ratio = chinese / len(text) if len(text) > 0 else 0
|
||||
return ratio < 0.10
|
||||
```
|
||||
|
||||
### 2.2 批量删除(通过 SQLite 直写)
|
||||
|
||||
停掉 calibre 进程后,通过 SQLite 删除记录并清理文件系统:
|
||||
|
||||
```sql
|
||||
DELETE FROM data WHERE book = ?
|
||||
DELETE FROM books_authors_link WHERE book = ?
|
||||
DELETE FROM books_languages_link WHERE book = ?
|
||||
DELETE FROM books_publishers_link WHERE book = ?
|
||||
DELETE FROM books_series_link WHERE book = ?
|
||||
DELETE FROM books_tags_link WHERE book = ?
|
||||
DELETE FROM comments WHERE book = ?
|
||||
DELETE FROM identifiers WHERE book = ?
|
||||
DELETE FROM books WHERE id = ?
|
||||
```
|
||||
|
||||
同时删除对应目录:`shutil.rmtree(os.path.join('/home/yangxuan/calibre/books', bpath))`
|
||||
|
||||
### 2.3 完整删除脚本
|
||||
|
||||
```python
|
||||
import sqlite3, os, shutil
|
||||
|
||||
db_path = '/home/yangxuan/calibre/books/metadata.db'
|
||||
conn = sqlite3.connect(db_path)
|
||||
c = conn.cursor()
|
||||
|
||||
c.execute('''
|
||||
SELECT b.id, b.title, b.path, d.name, d.uncompressed_size
|
||||
FROM books b
|
||||
JOIN data d ON d.book = b.id
|
||||
WHERE d.format = 'TXT'
|
||||
''')
|
||||
rows = c.fetchall()
|
||||
|
||||
to_delete = []
|
||||
|
||||
for bid, title, bpath, fname, size in rows:
|
||||
full_path = os.path.join('/home/yangxuan/calibre/books', bpath, fname + '.txt')
|
||||
|
||||
# 条件0:垃圾格式标题(Python列表、转义字符等)
|
||||
if is_garbage_format_title(title):
|
||||
to_delete.append((bid, bpath))
|
||||
continue
|
||||
|
||||
# 条件1:小文件垃圾
|
||||
if size < 10240:
|
||||
to_delete.append((bid, bpath))
|
||||
continue
|
||||
|
||||
if not os.path.exists(full_path):
|
||||
continue
|
||||
|
||||
# 条件2:内容损坏(中文 < 10%)
|
||||
if is_dirty_content(full_path):
|
||||
to_delete.append((bid, bpath))
|
||||
|
||||
print(f"待删除: {len(to_delete)}", flush=True)
|
||||
for bid, bpath in to_delete:
|
||||
for table in ['data', 'books_authors_link', 'books_languages_link',
|
||||
'books_publishers_link', 'books_series_link', 'books_tags_link',
|
||||
'comments', 'identifiers']:
|
||||
c.execute(f'DELETE FROM {table} WHERE book=?', (bid,))
|
||||
c.execute('DELETE FROM books WHERE id=?', (bid,))
|
||||
full_dir = os.path.join('/home/yangxuan/calibre/books', bpath)
|
||||
if os.path.exists(full_dir):
|
||||
shutil.rmtree(full_dir)
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print(f"完成,删除: {len(to_delete)}", flush=True)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、书名批量清洗(作者标记去除 + 前 20 中文截取)
|
||||
|
||||
对于书名中含 `作者:xxx`、`翻译:xxx` 标记但作者字段已有值的书,使用此脚本批量清洗:
|
||||
|
||||
```python
|
||||
import sqlite3, re
|
||||
|
||||
db_path = '/home/yangxuan/calibre/books/metadata.db'
|
||||
conn = sqlite3.connect(db_path)
|
||||
c = conn.cursor()
|
||||
|
||||
c.execute('''
|
||||
SELECT b.id, b.title, a.name as author_name, d.name as file_name
|
||||
FROM books b
|
||||
JOIN data d ON d.book = b.id AND d.format = 'TXT'
|
||||
JOIN books_authors_link l ON b.id = l.book
|
||||
JOIN authors a ON l.author = a.id
|
||||
WHERE (b.title LIKE '%作者:%' OR b.title LIKE '%作者:%' OR b.title LIKE '%翻译:%' OR b.title LIKE '%翻译:%')
|
||||
AND a.name NOT IN ('未知', 'Unknown')
|
||||
ORDER BY b.id
|
||||
''')
|
||||
rows = c.fetchall()
|
||||
|
||||
for bid, title, author_name, file_name in rows:
|
||||
# 1. 从 data.name 提取作者并更新 author 字段(如不同)
|
||||
name_clean = re.sub(r'\s*-\s*(Unknown|未知)\s*$', '', file_name)
|
||||
m = re.search(r'[ \s]*?(?:作者|翻译|原?作者)[::]\s*([^\s_)\)《\[【\]】\-,]+)', name_clean)
|
||||
if m:
|
||||
extracted = m.group(1).strip()
|
||||
if extracted and extracted != author_name and len(extracted) < 30:
|
||||
cur = conn.cursor()
|
||||
cur.execute("SELECT id FROM authors WHERE name=?", (extracted,))
|
||||
ar = cur.fetchone()
|
||||
if ar:
|
||||
aid = ar[0]
|
||||
else:
|
||||
cur.execute("INSERT INTO authors (name, sort) VALUES (?, ?)", (extracted, extracted.replace(',', '').strip()))
|
||||
aid = cur.lastrowid
|
||||
cur.execute("UPDATE books_authors_link SET author=? WHERE book=?", (aid, bid))
|
||||
cur.close()
|
||||
|
||||
# 2. 裁剪书名:去标记 + 前 20 中文
|
||||
clean_title = re.sub(r'[ \s]*?(?:作者|翻译|原?作者)[::].*$', '', file_name, flags=re.DOTALL)
|
||||
clean_title = re.sub(r'\s*-\s*(Unknown|未知)\s*$', '', clean_title).strip()
|
||||
|
||||
if clean_title:
|
||||
count = 0
|
||||
pos = 0
|
||||
for i, ch in enumerate(clean_title):
|
||||
if '\u4e00' <= ch <= '\u9fff':
|
||||
count += 1
|
||||
if count == 20:
|
||||
pos = i + 1
|
||||
break
|
||||
if count > 20:
|
||||
clean_title = clean_title[:pos]
|
||||
|
||||
if clean_title != title:
|
||||
c.execute("UPDATE books SET title=? WHERE id=?", (clean_title, bid))
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、常用查询
|
||||
|
||||
### 4.1 查看 XX 作者的书
|
||||
|
||||
```bash
|
||||
sqlite3 /home/yangxuan/calibre/books/metadata.db "
|
||||
SELECT b.id, b.title
|
||||
FROM books b JOIN books_authors_link l ON b.id = l.book
|
||||
JOIN authors a ON l.author = a.id
|
||||
WHERE a.name = 'Unknown'
|
||||
LIMIT 20;
|
||||
"
|
||||
```
|
||||
|
||||
### 4.2 统计作者分布
|
||||
|
||||
```bash
|
||||
sqlite3 /home/yangxuan/calibre/books/metadata.db "
|
||||
SELECT a.name, COUNT(*) as cnt
|
||||
FROM books b JOIN books_authors_link l ON b.id = l.book
|
||||
JOIN authors a ON l.author = a.id
|
||||
GROUP BY a.name ORDER BY cnt DESC LIMIT 20;
|
||||
"
|
||||
```
|
||||
|
||||
### 4.3 书名异常检测
|
||||
|
||||
```bash
|
||||
sqlite3 /home/yangxuan/calibre/books/metadata.db "
|
||||
SELECT id, title FROM books WHERE
|
||||
title LIKE '%作者%' OR title LIKE '%翻译%' OR
|
||||
title LIKE '%.com%' OR title LIKE '%http%' OR
|
||||
title LIKE '%内容简介%' OR
|
||||
LENGTH(title) > 80;
|
||||
"
|
||||
```
|
||||
|
||||
### 4.4 书库统计
|
||||
|
||||
```bash
|
||||
python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/home/yangxuan/calibre/books/metadata.db')
|
||||
c = conn.cursor()
|
||||
c.execute('SELECT COUNT(*) FROM books')
|
||||
print(f'总书籍: {c.fetchone()[0]}')
|
||||
c.execute('SELECT COUNT(*) FROM data WHERE format=\"TXT\"')
|
||||
print(f'TXT 文件: {c.fetchone()[0]}')
|
||||
conn.close()
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、操作流程(推荐顺序)
|
||||
|
||||
```bash
|
||||
cd /home/yangxuan/calibre
|
||||
|
||||
# 1. 停服务 + 备份
|
||||
docker compose down
|
||||
cp books/metadata.db books/metadata.db.backup.$(date +%Y%m%d)
|
||||
killall calibre 2>/dev/null
|
||||
|
||||
# 2. 脏数据清理(内容损坏 + 小文件 + 垃圾格式标题)
|
||||
python3 /path/to/clean_dirty_data.py
|
||||
|
||||
# 3. 书名清洗 & 作者提取(含 data.name 级别提取)
|
||||
calibre-debug /path/to/fix_titles.py
|
||||
|
||||
# 4. 检查是否有遗漏的异常书籍
|
||||
sqlite3 books/metadata.db \"SELECT id, title FROM books WHERE title LIKE '%作者%' OR title LIKE '%翻译%' OR title LIKE '%.com%';\" | head -10
|
||||
|
||||
# 5. 重启
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 六、已知问题
|
||||
|
||||
### `set_metadata` 报 `Errno 36 文件名过长`
|
||||
|
||||
当 `author_sort` 字段超 80 字节时,Calibre 路径会触及 ext4 255 字节限制。先清理极长作者名:
|
||||
|
||||
```bash
|
||||
sqlite3 /home/yangxuan/calibre/books/metadata.db "
|
||||
UPDATE authors SET name='Unknown' WHERE LENGTH(name) > 30;
|
||||
"
|
||||
```
|
||||
|
||||
### WSL `/mnt/f` 删除 NTFS 文件权限不够
|
||||
|
||||
WSL 的 drvfs 默认不带 metadata 选项,`sudo rm`、`os.remove`、`cmd.exe /c del` 均无法删除 NTFS 文件。不要在 WSL 下尝试跨分区删除操作,直接让 Windows 侧(文件资源管理器/PowerShell)操作。
|
||||
|
||||
### 书名长度截取规则
|
||||
|
||||
`title` 字段清洗后最多保留 20 个中文字符。计数规则:只计 Unicode 范围的 `\u4e00-\u9fff` 中文字符,保留第 20 个中文之前的所有字符(含非中文)。如书名不足 20 个中文则全保留。
|
||||
|
||||
### `title` vs `data.name` 区别
|
||||
|
||||
- `books.title`:Calibre 界面显示的书名(清洗后的)
|
||||
- `data.name`:导入时的原始文件名(不含 `.txt`)
|
||||
- 作者提取优先使用 `data.name`,其文件名保留原始信息、更可靠
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
name: "f-download-sorter"
|
||||
description: "整理 /mnt/f 下下载的视频文件:扫描→清垃圾→按规则归档到7个大类目录"
|
||||
---
|
||||
|
||||
# /mnt/f 下载文件批量归档整理
|
||||
|
||||
## 用途
|
||||
把 source_dirs 里堆叠的下载视频,按规则自动清理、归类到大类目录。
|
||||
|
||||
## Touch point
|
||||
用户说"整理 /mnt/f 视频""归档 collections""下载文件太多理一下"等,调用此 skill。
|
||||
|
||||
## 流程
|
||||
1. 运行 `scripts/scan_and_clean.py` → 扫描 + 清垃圾
|
||||
2. 运行 `scripts/classify_and_move.py` → 按规则归档
|
||||
3. 报告结果
|
||||
|
||||
## 配置
|
||||
核心配置在 `config.json`,可自定义:
|
||||
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| `source_dirs` | 源目录列表,默认 `["/mnt/f/collections", "/mnt/f/video"]` |
|
||||
| `keep_as_is` | 已有元数据不动的内容 |
|
||||
| `min_size_mb` | 小于此值且无实质视频的目录自动删除 |
|
||||
| `categories` | 分类规则数组,每个有 `dir`(目标路径)和 `keywords`(关键词列表) |
|
||||
| `fallback_dir` | 无关键词匹配时的兜底目录 |
|
||||
|
||||
### 添加新分类
|
||||
在 `categories` 数组中新增一项:
|
||||
```json
|
||||
{
|
||||
"dir": "/mnt/f/新分类",
|
||||
"keywords": ["关键词A", "关键词B"]
|
||||
}
|
||||
```
|
||||
|
||||
### 添加新源目录
|
||||
在 `source_dirs` 中追加路径。
|
||||
|
||||
### 保护不移动的内容
|
||||
在 `keep_as_is` 列表追加名称。
|
||||
|
||||
## 注意事项
|
||||
- 同一分区 mv,不复制数据
|
||||
- 不预览文件内容,只基于文件名关键词
|
||||
- WSL `/mnt/` 跨文件系统扫描慢,大数量时建议用 Windows 原生 Python 执行
|
||||
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"source_dirs": [
|
||||
"/mnt/f/collections",
|
||||
"/mnt/f/video"
|
||||
],
|
||||
"keep_as_is": ["重生之征服郭伯母"],
|
||||
"min_size_mb": 50,
|
||||
"categories": [
|
||||
{
|
||||
"dir": "/mnt/f/过那条巷子",
|
||||
"keywords": ["过那条巷子"]
|
||||
},
|
||||
{
|
||||
"dir": "/mnt/f/2048合集",
|
||||
"keywords": ["2048"]
|
||||
},
|
||||
{
|
||||
"dir": "/mnt/f/AI短剧",
|
||||
"keywords": ["AI短剧", "AI自制", "AI视频", "美丽新世界", "少年阿宾", "NTR寝取", "嫂子", "魔女小姨子", "桃运圣医", "我和老师的约定"]
|
||||
},
|
||||
{
|
||||
"dir": "/mnt/f/FC2福利姬",
|
||||
"keywords": ["FC2", "Comatozze", "Dainty", "KAKAE", "lananlanan", "P站"]
|
||||
},
|
||||
{
|
||||
"dir": "/mnt/f/合集种子",
|
||||
"keywords": ["BT-btt"]
|
||||
},
|
||||
{
|
||||
"dir": "/mnt/f/国产露脸",
|
||||
"keywords": ["19岁", "冯帆", "momocats", "露脸才是王道", "香蕉"]
|
||||
},
|
||||
{
|
||||
"dir": "/mnt/f/自拍泄漏",
|
||||
"keywords": ["泄密", "自拍", "自慰", "原相机", "情侣", "校花", "嫩妹", "福利姬", "裸聊", "绿帽", "良家", "约炮", "狗哥", "土豪", "原档", "维度", "桃子", "娇娇", "学妹", "反差"]
|
||||
}
|
||||
],
|
||||
"fallback_dir": "/mnt/f/AI短剧"
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
按规则分类归档 source_dirs 中的目录到 categories 定义的大类目录
|
||||
读配置文件:与脚本同目录下的 config.json
|
||||
"""
|
||||
import os, json, shutil
|
||||
|
||||
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
def load_config():
|
||||
cfg_path = os.path.join(SCRIPT_DIR, "..", "config.json")
|
||||
with open(cfg_path) as f:
|
||||
return json.load(f)
|
||||
|
||||
def classify(name, cfg):
|
||||
"""返回目标目录路径,或 None(不移动)"""
|
||||
if name in cfg.get("keep_as_is", []):
|
||||
return None
|
||||
for cat in cfg["categories"]:
|
||||
keywords = cat["keywords"]
|
||||
if any(k in name for k in keywords):
|
||||
return cat["dir"]
|
||||
fallback = cfg.get("fallback_dir")
|
||||
if fallback:
|
||||
return fallback
|
||||
return None
|
||||
|
||||
def main():
|
||||
cfg = load_config()
|
||||
src_dirs = cfg["source_dirs"]
|
||||
|
||||
moves = []
|
||||
for base in src_dirs:
|
||||
if not os.path.isdir(base):
|
||||
print(f" [WARN] 目录不存在: {base}")
|
||||
continue
|
||||
for entry in sorted(os.listdir(base)):
|
||||
full = os.path.join(base, entry)
|
||||
if not os.path.isdir(full):
|
||||
continue
|
||||
dst_base = classify(entry, cfg)
|
||||
if dst_base is None:
|
||||
print(f" ○ 不动: {entry}")
|
||||
continue
|
||||
dst = os.path.join(dst_base, entry)
|
||||
if os.path.exists(dst):
|
||||
i = 1
|
||||
while os.path.exists(f"{dst}_{i}"): i += 1
|
||||
dst = f"{dst}_{i}"
|
||||
moves.append((full, dst))
|
||||
|
||||
# 创建目录
|
||||
for _, d in moves:
|
||||
os.makedirs(os.path.dirname(d), exist_ok=True)
|
||||
|
||||
# 执行移动
|
||||
print(f"开始移动 {len(moves)} 个条目...")
|
||||
for src, dst in moves:
|
||||
try:
|
||||
shutil.move(src, dst)
|
||||
print(f" ✓ {os.path.basename(src)}")
|
||||
except Exception as e:
|
||||
print(f" ✗ {os.path.basename(src)} -> {e}")
|
||||
|
||||
# 清理空源目录
|
||||
for d in src_dirs:
|
||||
if os.path.isdir(d) and not os.listdir(d):
|
||||
os.rmdir(d)
|
||||
print(f" ✓ 已删空目录: {d}")
|
||||
|
||||
# 报告
|
||||
base = os.path.commonpath([os.path.dirname(d) for _, d in moves])
|
||||
for _, d in moves:
|
||||
base = os.path.commonpath([base, os.path.dirname(d)])
|
||||
base = base or "/"
|
||||
|
||||
print(f"\n=== 归档完成 ===")
|
||||
seen = set()
|
||||
for _, d in moves:
|
||||
parent = os.path.dirname(d)
|
||||
if parent not in seen:
|
||||
sz = 0
|
||||
cnt = 0
|
||||
for dp, dn, fn in os.walk(parent):
|
||||
for f in fn:
|
||||
cnt += 1
|
||||
try: sz += os.path.getsize(os.path.join(dp, f))
|
||||
except: pass
|
||||
print(f" {parent}/ ({cnt} 文件, {sz/1024**3:.1f}GB)")
|
||||
seen.add(parent)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,67 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
扫描 + 清理垃圾
|
||||
遍历 source_dirs,清理空壳目录和小视频残片
|
||||
读配置文件:与脚本同目录下的 config.json
|
||||
"""
|
||||
import os, json, sys
|
||||
|
||||
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
def load_config():
|
||||
cfg_path = os.path.join(SCRIPT_DIR, "..", "config.json")
|
||||
with open(cfg_path) as f:
|
||||
return json.load(f)
|
||||
|
||||
def scan_and_clean(cfg):
|
||||
src_dirs = cfg["source_dirs"]
|
||||
min_size = cfg["min_size_mb"] * 1024 * 1024
|
||||
video_exts = {'.mp4','.mkv','.avi','.mov','.wmv','.flv','.webm','.ts','.m4v','.rmvb'}
|
||||
|
||||
targets = []
|
||||
print("=== 扫描 ===")
|
||||
for base in src_dirs:
|
||||
if not os.path.isdir(base):
|
||||
print(f" [WARN] 目录不存在: {base}")
|
||||
continue
|
||||
for entry in sorted(os.listdir(base)):
|
||||
full = os.path.join(base, entry)
|
||||
if not os.path.isdir(full):
|
||||
continue
|
||||
total_sz = 0
|
||||
real_video = 0
|
||||
try:
|
||||
for dp, dn, fn in os.walk(full):
|
||||
for f in fn:
|
||||
fp = os.path.join(dp, f)
|
||||
try:
|
||||
total_sz += os.path.getsize(fp)
|
||||
except: pass
|
||||
if os.path.splitext(f)[1].lower() in video_exts:
|
||||
real_video += 1
|
||||
except: pass
|
||||
|
||||
action = "KEEP"
|
||||
if total_sz < min_size and real_video == 0:
|
||||
action = "DELETE"
|
||||
elif total_sz < min_size and real_video > 0:
|
||||
action = "DELETE"
|
||||
|
||||
print(f" [{action:>6}] {entry} ({total_sz/1024**2:.0f}MB, {real_video}视频)")
|
||||
|
||||
if action == "DELETE":
|
||||
targets.append(full)
|
||||
|
||||
if targets:
|
||||
print(f"\n待删除: {len(targets)} 个")
|
||||
print("开始删除...")
|
||||
import shutil
|
||||
for d in targets:
|
||||
shutil.rmtree(d)
|
||||
print(f" ✓ 已删: {os.path.basename(d)}")
|
||||
else:
|
||||
print("\n无需清理。")
|
||||
|
||||
if __name__ == "__main__":
|
||||
cfg = load_config()
|
||||
scan_and_clean(cfg)
|
||||
@@ -0,0 +1,166 @@
|
||||
---
|
||||
name: "jav-media-sorter"
|
||||
description: "JAV 视频整理:组装文件、按演员分组、刮削 nfo 元数据与封面图"
|
||||
---
|
||||
|
||||
# JAV 媒体整理与元数据刮削技能
|
||||
|
||||
将 qbittorrent 下载完成的 JAV 视频( mp4 + srt + nfo )文件整理成 `/mnt/f/av/` 格式,按演员分类,并尝试刮削封面图。
|
||||
|
||||
## 目录结构规范
|
||||
|
||||
目标格式(参考 `/mnt/f/av/`):
|
||||
|
||||
```
|
||||
/mnt/f/av/
|
||||
├── 演员名/
|
||||
│ ├── 番号-C 演员名/ ← 如果已有番号-C 目录则保留原名
|
||||
│ │ ├── 番号.mp4
|
||||
│ │ ├── 番号.chs.srt
|
||||
│ │ ├── 番号.nfo
|
||||
│ │ ├── poster.jpg
|
||||
│ │ └── fanart.jpg
|
||||
│ └── ...
|
||||
├── 未知演员/
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
另一套目录结构(参考 `/mnt/e/01_automated/日本电影/`):
|
||||
|
||||
```
|
||||
/mnt/e/01_automated/日本电影/
|
||||
├── 演员名/
|
||||
│ ├── 番号 演员名/
|
||||
│ │ ├── 番号.mp4
|
||||
│ │ ├── 番号.nfo
|
||||
│ │ ├── 番号-poster.jpg
|
||||
│ │ └── 番号-thumb.jpg
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
## 前置条件
|
||||
|
||||
- Clash 代理运行在 Docker(端口 7443,认证 `proxy:5gynj20J`)
|
||||
- FlareSolverr 运行在 Docker(已下线,按需启动)
|
||||
- Python3(用于 nfo 和整理脚本)
|
||||
- Docker(FlareSolverr + Clash)
|
||||
|
||||
## 操作流程
|
||||
|
||||
### 1. 检查下载目录
|
||||
|
||||
```bash
|
||||
ls /mnt/f/qbittorrent/failed/
|
||||
```
|
||||
|
||||
通常结构是 `failed/` 目录下平铺所有 mp4、srt(.chs.srt)、nfo 文件。
|
||||
|
||||
### 2. 按演员创建目标文件夹
|
||||
|
||||
在 `failed/` 目录内创建 `番号-C 演员名/` 文件夹,方便手动拖到 `/mnt/f/av/` 中对应演员目录。
|
||||
|
||||
```bash
|
||||
cd /mnt/f/qbittorrent/failed
|
||||
mkdir -p "ADN-016-C 明日見未来"
|
||||
# ... 以此类推
|
||||
```
|
||||
|
||||
### 3. 按演员分组(移动目录到演员子目录)
|
||||
|
||||
若目录直接放在 `/mnt/f/av/` 根目录下,需要移动到对应演员子目录:
|
||||
|
||||
```bash
|
||||
cd /mnt/f/av
|
||||
mkdir -p "明日見未来"
|
||||
mv "ADN-016-C 明日見未来" "明日見未来/"
|
||||
# ... 以此类推
|
||||
```
|
||||
|
||||
### 4. 读取 nfo 获取演员信息
|
||||
|
||||
每个作品的 nfo 文件包含元数据,用 Python 解析:
|
||||
|
||||
```python
|
||||
import xml.etree.ElementTree as ET
|
||||
tree = ET.parse('番号.nfo')
|
||||
root = tree.getroot()
|
||||
actors = [a.find('name').text for a in root.findall('actor')]
|
||||
dvd_id = root.find('num').text
|
||||
```
|
||||
|
||||
### 5. 封面图刮削
|
||||
|
||||
尝试从以下来源下载封面图:
|
||||
|
||||
#### 5.1 FlareSolverr + Clash 代理(首选)
|
||||
|
||||
```bash
|
||||
FLARE="http://127.0.0.1:8191/v1"
|
||||
CLASH_PROXY="http://proxy:***@host.docker.internal:7443"
|
||||
|
||||
# FlareSolverr API 请求格式
|
||||
curl -s "$FLARE" -d '{
|
||||
"cmd":"request.get",
|
||||
"url":"https://javdb.com/search?q=番号&f=all",
|
||||
"maxTimeout":45000,
|
||||
"proxy":{"url":"'$CLASH_PROXY'"}
|
||||
}' -H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
从返回的 JSON 中提取 `solution.response` 中的 HTML,解析出:
|
||||
|
||||
- 详情页链接 `href="/v/xxx"`
|
||||
- 详情页中 `class="video-cover"` 的 `src` 属性(封面图片 URL)
|
||||
|
||||
然后通过 Clash 代理下载图片:
|
||||
|
||||
```bash
|
||||
curl -x "http://proxy:***@127.0.0.1:7443" -s -L -o "poster.jpg" \
|
||||
-H "Referer: https://javdb.com/" "$POSTER_URL"
|
||||
```
|
||||
|
||||
#### 5.2 备用:Python socks5 直连
|
||||
|
||||
```python
|
||||
import socks, socket
|
||||
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 7443,
|
||||
username="proxy", password="***", rdns=True)
|
||||
socket.socket = socks.socksocket
|
||||
import urllib.request, re
|
||||
req = urllib.request.Request("https://javdb.com", headers={
|
||||
"User-Agent": "Mozilla/5.0"})
|
||||
```
|
||||
|
||||
#### 5.3 已知限制
|
||||
|
||||
- JavDB 有 Cloudflare 反爬,Python `requests`/`cloudscraper` 均可能被拦截
|
||||
- FlareSolverr 走 HTTP 代理时可能无法完成 Cloudflare JS 挑战
|
||||
- FANZA/DMM 图片 CDN 返回小图(~3KB),真实大图需 JS 渲染
|
||||
- 最可靠方案:**Windows 端浏览器手动下载**
|
||||
|
||||
### 6. 图片命名规则
|
||||
|
||||
- `/mnt/f/av/` 格式: `poster.jpg` + `fanart.jpg`(两图相同)
|
||||
- `/mnt/e/01_automated/日本电影/` 格式: `番号-poster.jpg` + `番号-thumb.jpg`
|
||||
|
||||
## 注意事项
|
||||
|
||||
- **WSL 操作 /mnt 文件极慢**:大量小文件操作应在 Windows 原生 Python 中执行
|
||||
- **代理密码注意 shell 展开**:Clash 密码 `5gynj20J` 在 bash 中写为 `"proxy:*"`(用双引号避免 `***` 展开)
|
||||
- **FlareSolverr 密码兼容性**:SOCKS5 代理认证不支持通过环境变量传递。HTTP 代理用 `proxy` 参数
|
||||
- **Clash 端口**:mixed-port 7443,同时支持 HTTP 和 SOCKS5
|
||||
- **删除冗余的字幕目录**:`*-中文字幕/` 目录中的 `.srt` 与 `failed/` 目录中的 `.chs.srt` 多数是重复文件(MD5 一致)
|
||||
|
||||
## 依赖检查
|
||||
|
||||
```bash
|
||||
# FlareSolverr
|
||||
docker ps | grep flaresolverr
|
||||
|
||||
# Clash
|
||||
docker ps | grep clash
|
||||
|
||||
# Python
|
||||
python3 --version
|
||||
pip3 list 2>/dev/null | grep -i cloudscraper
|
||||
```
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
name: "jellyfin-nfo-builder"
|
||||
description: "为剧集整理统一文件名、生成缺失的 Jellyfin .nfo 剧集元数据"
|
||||
---
|
||||
|
||||
# Jellyfin NFO 剧集元数据补齐技能
|
||||
|
||||
为剧集目录整理统一文件名,并为缺失的集数生成标准的 Jellyfin `.nfo` 元数据文件。
|
||||
|
||||
## 适用场景
|
||||
|
||||
- 手动清理/重命名剧集文件后,部分集数的 `.nfo` 元数据缺失
|
||||
- 从不同来源合并剧集(高质量版 + 补全集),需要统一格式
|
||||
- 批量标准化剧集视频文件和元数据
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 视频文件已整理到一个目录
|
||||
- 有一条完整的剧集(有 `.nfo` 示例可参照格式)
|
||||
- 了解剧集的总集数
|
||||
- shell 工作目录为剧集所在目录
|
||||
|
||||
## 步骤
|
||||
|
||||
### 1. 检查现有状态
|
||||
|
||||
列出目录下所有视频文件和 `.nfo` 文件:
|
||||
|
||||
```bash
|
||||
ls -lh *.mp4 *.nfo 2>/dev/null
|
||||
```
|
||||
|
||||
确定:
|
||||
- 总集数
|
||||
- 已有哪些集的 `.nfo`
|
||||
- 哪些集缺失 `.nfo`
|
||||
- 文件名是否统一
|
||||
|
||||
### 2. 读取一个已有 `.nfo` 作为模板
|
||||
|
||||
```bash
|
||||
cat "剧集名 第xx集.nfo"
|
||||
```
|
||||
|
||||
标准 Jellyfin 剧集 nfo 格式:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<episodedetails>
|
||||
<title>第N集</title>
|
||||
<season>1</season>
|
||||
<episode>N</episode>
|
||||
<plot>剧集名 第N集</plot>
|
||||
<runtime>233</runtime>
|
||||
<aired>2026-01-01</aired>
|
||||
</episodedetails>
|
||||
```
|
||||
|
||||
### 3. 生成缺失的 `.nfo`
|
||||
|
||||
```bash
|
||||
cd /path/to/剧集目录
|
||||
for i in 4 5 8 9 10 11; do
|
||||
cat > "剧集名 第${i}集.nfo" << EOF
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<episodedetails>
|
||||
<title>第${i}集</title>
|
||||
<season>1</season>
|
||||
<episode>${i}</episode>
|
||||
<plot>剧集名 第${i}集</plot>
|
||||
<runtime>233</runtime>
|
||||
<aired>2026-01-01</aired>
|
||||
</episodedetails>
|
||||
EOF
|
||||
done
|
||||
```
|
||||
|
||||
> `runtime`(时长秒数)和 `aired`(首播日期)可以参照已有文件填写,或设为默认值。
|
||||
|
||||
### 4. 检查结果
|
||||
|
||||
```bash
|
||||
ls -lh *.mp4 *.nfo 2>/dev/null
|
||||
```
|
||||
|
||||
确认每一集都有对应的视频文件和 `.nfo` 文件。
|
||||
|
||||
## 注意事项
|
||||
|
||||
- Jellyfin 依赖文件名排序,统一格式为 `剧集名 第XX集.mp4`(两位数字,按字母排序正确)
|
||||
- `.nfo` 中的 `<episode>` 标签必须与集数一致
|
||||
- 如果整个剧集目录还没有 `tvshow.nfo` 和 `season.nfo`,也需要一并创建
|
||||
- 视频文件命名应与 `.nfo` 文件名完全一致(仅扩展名不同)
|
||||
Reference in New Issue
Block a user