3.21
This commit is contained in:
parent
eab6599075
commit
8bec7f4e65
118
README.md
118
README.md
@ -1,13 +1,26 @@
|
||||
## 同步本地Markdown至Typecho站点
|
||||
|
||||
场景：本人喜欢在本地用Typora写markdown文件，但又想同时同步至Typecho发表成文章；且由于md文件并不是一成不变的，经常需要对各个文件缝缝补补，要能实现本地更新/同步至博客更新。
|
||||
本项目基于https://github.com/Sundae97/typecho-markdown-file-publisher
|
||||
|
||||
亲测适配：Typecho1.2 php7.4.33
|
||||
实现效果：
|
||||
|
||||
- [x] 将markdown发布到typecho
|
||||
- [x] 发布前将markdown的图片资源上传到TencentCloud的COS中, 并替换markdown中的图片链接
|
||||
- [x] 将md所在的文件夹名称作为post的category(mysql发布可以插入category, xmlrpc接口暂时不支持category操作)
|
||||
- [x] 以title和category作为文章的唯一标识，如果数据库中已有该数据，将会更新现有文章，否则新增文章。
|
||||
|
||||
环境：Typecho1.2.1 php7.4.33
|
||||
|
||||
### 项目目录
|
||||
|
||||
|
||||
|
||||
`typecho_markdown_upload/main.py`是上传md文件到站点的核心脚本
|
||||
|
||||
`transfer_md/transfer.py`是对md文件进行预处理的脚本。
|
||||
|
||||
|
||||
|
||||
### **核心思路**
|
||||
|
||||
**一、预先准备**
|
||||
@ -58,7 +71,7 @@ github地址：[icret/EasyImages2.0: 简单图床 - 一款功能强大无数据
|
||||
|
||||
**文件结构统一**：
|
||||
|
||||
```
|
||||
```text
|
||||
md_files
|
||||
├── category1
|
||||
│ ├── file1.md
|
||||
@ -94,7 +107,7 @@ md_files
|
||||
|
||||
**初始化本地仓库**：
|
||||
|
||||
```
|
||||
```text
|
||||
git init
|
||||
```
|
||||
|
||||
@ -102,26 +115,26 @@ git init
|
||||
|
||||
将远程仓库地址添加为 `origin`（请将 `http://xxx` 替换为你的实际仓库地址）：
|
||||
|
||||
```
|
||||
```text
|
||||
git remote add origin http://xxx
|
||||
```
|
||||
|
||||
**添加文件并提交**：
|
||||
|
||||
```
|
||||
```text
|
||||
git add .
|
||||
git commit -m "Initial commit"
|
||||
```
|
||||
|
||||
**推送到远程仓库：**
|
||||
|
||||
```
|
||||
```text
|
||||
git push -u origin master
|
||||
```
|
||||
|
||||
**后续更新：**
|
||||
**后续更新（可写个.bat批量执行）：**
|
||||
|
||||
```
|
||||
```text
|
||||
git add .
|
||||
git commit -m "更新了xxx内容"
|
||||
git push
|
||||
@ -133,11 +146,11 @@ git push
|
||||
|
||||
**1. 确保脚本能够连接到 Typecho 使用的数据库**
|
||||
|
||||
本博客使用 docker-compose 部署 Typecho（参考：[【好玩儿的Docker项目】10分钟搭建一个Typecho博客｜太破口！念念不忘，必有回响！-我不是咕咕鸽](https://blog.laoda.de/archives/docker-compose-install-typecho)）。为了让脚本能访问 Typecho 的数据库，我将 Python 应用也通过 docker-compose 部署，这样所有服务均在同一网络中，互相之间可以直接通信。
|
||||
本博客使用 docker-compose 部署 Typecho（参考：[【好玩儿的Docker项目】10分钟搭建一个Typecho博客｜太破口！念念不忘，必有回响！-我不是咕咕鸽](https://blog.laoda.de/archives/docker-compose-install-typecho)）。为了让脚本能访问 Typecho 的数据库，我将 Python 应用pyapp也通过 docker-compose 部署，这样所有服务均在同一网络中，互相之间可以直接通信。
|
||||
|
||||
参考docker-compose.yml如下：
|
||||
|
||||
```
|
||||
```text
|
||||
services:
|
||||
nginx:
|
||||
image: nginx
|
||||
@ -171,6 +184,8 @@ services:
|
||||
pyapp:
|
||||
build: ./markdown_operation # Dockerfile所在的目录
|
||||
restart: "no"
|
||||
volumes:
|
||||
- /home/zy123/md_files:/markdown_operation/md_files
|
||||
networks:
|
||||
- web
|
||||
env_file:
|
||||
@ -201,9 +216,13 @@ networks:
|
||||
|
||||
|
||||
|
||||
**2. 将版本控制的 md_files 仓库克隆到 markdown_operation 目录中**
|
||||
**2. 将 `md_files` 挂载到容器中，保持最新内容同步**
|
||||
|
||||
确保在容器内可以直接访问到 md_files 内容，因此我们将使用 Git 进行版本控制的 md_files 仓库克隆到 markdown_operation 内部。这样，无论是执行脚本还是其他操作，都能轻松访问和更新 Markdown 文件。
|
||||
这样有几个优势：
|
||||
|
||||
- 不需要每次构建镜像或进入容器手动拉取；
|
||||
- 本地更新 `md_files` 后，容器内自动同步，无需额外操作；
|
||||
- 保持了宿主机上的 Git 版本控制和容器内的数据一致性。
|
||||
|
||||
|
||||
|
||||
@ -211,61 +230,100 @@ networks:
|
||||
|
||||
`pyapp`是本Python应用在容器内的名称。
|
||||
|
||||
构建镜像：
|
||||
1.构建镜像：
|
||||
|
||||
```
|
||||
```text
|
||||
docker-compose build pyapp
|
||||
```
|
||||
|
||||
启动容器并进入 Bash：
|
||||
2.启动容器并进入 Bash：
|
||||
|
||||
```
|
||||
```text
|
||||
docker-compose run --rm -it pyapp /bin/bash
|
||||
```
|
||||
|
||||
在容器内运行脚本：
|
||||
3.在容器内运行脚本：
|
||||
|
||||
```
|
||||
```text
|
||||
python typecho_markdown_upload/main.py
|
||||
```
|
||||
|
||||
2、3两步可合并为：
|
||||
|
||||
```text
|
||||
docker-compose run --rm pyapp python typecho_markdown_upload/main.py
|
||||
```
|
||||
|
||||
|
||||
|
||||
此时可以打开博客验证一下是否成功发布文章了！
|
||||
|
||||
**如果失败，可以验证mysql数据库:**
|
||||
|
||||
1️⃣ 进入 MySQL 容器：
|
||||
|
||||
```
|
||||
docker compose exec mysql mysql -uroot -p
|
||||
```text
|
||||
docker-compose exec mysql mysql -uroot -p
|
||||
# 输入你的 root 密码
|
||||
```
|
||||
|
||||
2️⃣ 切换到 Typecho 数据库并列出表：
|
||||
|
||||
```
|
||||
```text
|
||||
USE typecho;
|
||||
SHOW TABLES;
|
||||
```
|
||||
|
||||
3️⃣ 查看 `typecho_contents` 表结构（文章表）：
|
||||
|
||||
```
|
||||
```text
|
||||
DESCRIBE typecho_contents;
|
||||
SHOW CREATE TABLE typecho_contents\G
|
||||
```
|
||||
|
||||
```text
|
||||
mysql> DESCRIBE typecho_contents;
|
||||
+--------------+------------------+------+-----+---------+----------------+
|
||||
| Field | Type | Null | Key | Default | Extra |
|
||||
+--------------+------------------+------+-----+---------+----------------+
|
||||
| cid | int(10) unsigned | NO | PRI | NULL | auto_increment |
|
||||
| title | varchar(150) | YES | | NULL | |
|
||||
| slug | varchar(150) | YES | UNI | NULL | |
|
||||
| created | int(10) unsigned | YES | MUL | 0 | |
|
||||
| modified | int(10) unsigned | YES | | 0 | |
|
||||
| text | longtext | YES | | NULL | |
|
||||
| order | int(10) unsigned | YES | | 0 | |
|
||||
| authorId | int(10) unsigned | YES | | 0 | |
|
||||
| template | varchar(32) | YES | | NULL | |
|
||||
| type | varchar(16) | YES | | post | |
|
||||
| status | varchar(16) | YES | | publish | |
|
||||
| password | varchar(32) | YES | | NULL | |
|
||||
| commentsNum | int(10) unsigned | YES | | 0 | |
|
||||
| allowComment | char(1) | YES | | 0 | |
|
||||
| allowPing | char(1) | YES | | 0 | |
|
||||
| allowFeed | char(1) | YES | | 0 | |
|
||||
| parent | int(10) unsigned | YES | | 0 | |
|
||||
| views | int(11) | YES | | 0 | |
|
||||
| agree | int(11) | YES | | 0 | |
|
||||
+--------------+------------------+------+-----+---------+----------------+
|
||||
```
|
||||
|
||||
4️⃣ 查询当前文章数量（确认执行前后有无变化）：
|
||||
|
||||
```
|
||||
```text
|
||||
SELECT COUNT(*) AS cnt FROM typecho_contents;
|
||||
```
|
||||
|
||||
|
||||
|
||||
### **自动化**
|
||||
|
||||
1.windows下写脚本自动/手动提交每日更新
|
||||
|
||||
2.远程仓库监测到更新自动实现钩子脚本,更新md_files并执行脚本
|
||||
|
||||
|
||||
|
||||
### TODO
|
||||
|
||||
- [x] 将markdown发布到typecho
|
||||
- [x] 发布前将markdown的图片资源上传到TencentCloud的COS中, 并替换markdown中的图片链接
|
||||
- [x] 将md所在的文件夹名称作为post的category(mysql发布可以插入category, xmlrpc接口暂时不支持category操作)
|
||||
- [ ] category的层级
|
||||
- [ ] 发布前先获取所有post信息, 不发布已经发布过的post
|
||||
- [ ] typecho_contents表中的slug字段代表链接中的日志缩略名，如wordpress风格 `/archives/{slug}.html`，目前是默认int自增，有需要的话可以在插入文章时手动设置该字段。
|
||||
|
||||
|
||||
@ -194,6 +194,35 @@ def process_md_file_remote(md_file):
|
||||
f.write(content)
|
||||
print(f"已更新: {md_file}")
|
||||
|
||||
def add_language_to_codeblocks(filepath, language="text"):
|
||||
'''
|
||||
若代码块没有指定语言，Typera中能正常显示，但是博客中的markdown解析器通常会错误显示。
|
||||
'''
|
||||
with open(filepath, 'r', encoding='utf-8') as f:
|
||||
lines = f.readlines()
|
||||
|
||||
in_code_block = False
|
||||
new_lines = []
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
# 判断是否为代码块标记
|
||||
if stripped.startswith("```"):
|
||||
if not in_code_block:
|
||||
# 当前为代码块的起始位置
|
||||
# 如果这一行只有三个反引号，则添加语言参数
|
||||
if stripped == "```":
|
||||
# 这里使用replace仅替换第一次出现的```，保留原有的换行符和空白
|
||||
line = line.replace("```", f"```{language}", 1)
|
||||
in_code_block = True
|
||||
else:
|
||||
# 当前为代码块的结束位置，不添加语言
|
||||
in_code_block = False
|
||||
new_lines.append(line)
|
||||
|
||||
# 将修改后的内容写回文件（也可选择写入新文件）
|
||||
with open(filepath, 'w', encoding='utf-8') as f:
|
||||
f.writelines(new_lines)
|
||||
|
||||
|
||||
def scan_files(base_folder, exclude_folders):
|
||||
"""
|
||||
@ -218,6 +247,7 @@ def process_md_files(input_path, output_path, type, exclude_folders=None):
|
||||
type == 1: process_md_file_local
|
||||
type == 2: process_md_file_with_assets
|
||||
type == 3: process_md_file_remote
|
||||
type == 4:
|
||||
"""
|
||||
# 创建输出目录（如果不存在）
|
||||
os.makedirs(output_path, exist_ok=True)
|
||||
@ -235,10 +265,12 @@ def process_md_files(input_path, output_path, type, exclude_folders=None):
|
||||
process_md_file_with_assets(md_file, output_path) # url改为本地，assets方式，图片和md文件都存output_path
|
||||
elif type == 3:
|
||||
process_md_file_remote(md_file) # 图片url改为公网链接
|
||||
elif type == 4:
|
||||
add_language_to_codeblocks(md_file)
|
||||
else:
|
||||
print(f"未知的处理类型: {type}")
|
||||
|
||||
print("处理完成！所有图片已保存至:", os.path.abspath(output_path))
|
||||
print("该文件夹下的md_files已全部处理完成！", os.path.abspath(input_path))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
@ -247,14 +279,12 @@ if __name__ == "__main__":
|
||||
try:
|
||||
type_value = int(sys.argv[1])
|
||||
except ValueError:
|
||||
print("第一个参数必须为整数，表示处理类型（1, 2 或 3）")
|
||||
print("第一个参数必须为整数，表示处理类型（1, 2 , 3, 4）")
|
||||
sys.exit(1)
|
||||
else:
|
||||
type_value = 3
|
||||
type_value = 4
|
||||
|
||||
# 这里的输入输出路径根据实际情况修改
|
||||
# input_path = os.getenv('BASE_FOLDER')
|
||||
input_path=r'D:\folder\study\md_files'
|
||||
# output_path = os.getenv('OUTPUT_FOLDER')
|
||||
output_path=r'D:\folder\study\md_files\output'
|
||||
input_path = os.getenv('BASE_FOLDER') #docker环境
|
||||
output_path = os.getenv('OUTPUT_FOLDER')
|
||||
process_md_files(input_path, output_path, type_value)
|
||||
|
||||
@ -23,7 +23,7 @@ class TypechoDirectMysqlPublisher:
|
||||
初始化分类列表到 self.__exist_categories
|
||||
"""
|
||||
cursor = self.__db.cursor()
|
||||
sql = "SELECT mid, name FROM %s WHERE type='category'" % self.__categories_table_name
|
||||
sql = f"SELECT mid, name FROM {self.__categories_table_name} WHERE type='category'"
|
||||
cursor.execute(sql)
|
||||
results = cursor.fetchall()
|
||||
self.__exist_categories = []
|
||||
@ -45,11 +45,10 @@ class TypechoDirectMysqlPublisher:
|
||||
"""
|
||||
cursor = self.__db.cursor()
|
||||
sql = (
|
||||
"INSERT INTO %s "
|
||||
f"INSERT INTO {self.__categories_table_name} "
|
||||
"(`name`, `slug`, `type`, `description`, `count`, `order`, `parent`) "
|
||||
"VALUES "
|
||||
"('%s', '%s', 'category', '', 0, 1, 0)"
|
||||
) % (self.__categories_table_name, category_name, category_name)
|
||||
f"VALUES ('{category_name}', '{category_name}', 'category', '', 0, 1, 0)"
|
||||
)
|
||||
cursor.execute(sql)
|
||||
mid = cursor.lastrowid
|
||||
self.__db.commit()
|
||||
@ -63,11 +62,10 @@ class TypechoDirectMysqlPublisher:
|
||||
在 typecho_relationships 中插入文章与分类的关联
|
||||
"""
|
||||
insert_relationship_sql = (
|
||||
"INSERT INTO %s "
|
||||
"(`cid`, `mid`) "
|
||||
"VALUES "
|
||||
"(%d, %d)"
|
||||
) % (self.__relationships_table_name, cid, mid)
|
||||
f"INSERT INTO {self.__relationships_table_name} "
|
||||
f"(`cid`, `mid`) "
|
||||
f"VALUES ({cid}, {mid})"
|
||||
)
|
||||
cursor.execute(insert_relationship_sql)
|
||||
|
||||
def __update_category_count(self, cursor, mid):
|
||||
@ -75,14 +73,16 @@ class TypechoDirectMysqlPublisher:
|
||||
分类下文章数 +1
|
||||
"""
|
||||
update_category_count_sql = (
|
||||
"UPDATE %s SET `count`=`count`+1 WHERE mid=%d"
|
||||
) % (self.__categories_table_name, mid)
|
||||
f"UPDATE {self.__categories_table_name} SET `count`=`count`+1 WHERE mid={mid}"
|
||||
)
|
||||
cursor.execute(update_category_count_sql)
|
||||
|
||||
def publish_post(self, title, content, category):
|
||||
"""
|
||||
如果 (category, title) 已存在 → 更新旧文章
|
||||
否则 → 插入新文章
|
||||
如果 (category, title) 已存在，则对比旧内容与新内容：
|
||||
- 若内容相同，则跳过更新
|
||||
- 若内容不同，则更新
|
||||
否则插入新文章
|
||||
"""
|
||||
cursor = self.__db.cursor()
|
||||
|
||||
@ -92,67 +92,53 @@ class TypechoDirectMysqlPublisher:
|
||||
mid = self.__add_category(category)
|
||||
|
||||
# 2. 查找同一分类下，是否已存在相同 title 的文章
|
||||
check_sql = """
|
||||
SELECT c.cid
|
||||
FROM %s c
|
||||
JOIN %s r ON c.cid = r.cid
|
||||
WHERE c.title = '%s'
|
||||
AND r.mid = %d
|
||||
check_sql = f"""
|
||||
SELECT c.cid, c.text
|
||||
FROM {self.__contents_table_name} c
|
||||
JOIN {self.__relationships_table_name} r ON c.cid = r.cid
|
||||
WHERE c.title = '{escape_string(title)}'
|
||||
AND r.mid = {mid}
|
||||
LIMIT 1
|
||||
""" % (
|
||||
self.__contents_table_name,
|
||||
self.__relationships_table_name,
|
||||
escape_string(title),
|
||||
mid
|
||||
)
|
||||
"""
|
||||
cursor.execute(check_sql)
|
||||
exist_row = cursor.fetchone()
|
||||
|
||||
now_time_int = int(time.time())
|
||||
content = '<!--markdown-->' + content
|
||||
content_with_mark = '<!--markdown-->' + content # 新内容加上标记
|
||||
escaped_content = escape_string(content_with_mark)
|
||||
|
||||
if exist_row:
|
||||
# ========== 执行更新逻辑 ==========
|
||||
cid = exist_row[0]
|
||||
update_sql = """
|
||||
UPDATE %s
|
||||
SET modified=%d,
|
||||
text='%s'
|
||||
WHERE cid=%d
|
||||
""" % (
|
||||
self.__contents_table_name,
|
||||
now_time_int,
|
||||
escape_string(content),
|
||||
cid
|
||||
)
|
||||
cursor.execute(update_sql)
|
||||
old_content = exist_row[1] or ""
|
||||
|
||||
# 如果你需要修改 slug、authorId、status 等字段，可在这里加上
|
||||
# 不需要改 relationships (分类关系) 和分类计数，因为 category 没变
|
||||
|
||||
print(f"[INFO] 更新文章成功: title={title}, cid={cid}, category={category}")
|
||||
if old_content.strip() == content_with_mark.strip():
|
||||
# 内容相同，不更新
|
||||
print(f"[INFO] 文章已存在且内容未修改: title={title}, cid={cid}, category={category}，跳过更新。")
|
||||
return cid
|
||||
else:
|
||||
# 内容不同，执行更新
|
||||
update_sql = f"""
|
||||
UPDATE {self.__contents_table_name}
|
||||
SET modified={now_time_int},
|
||||
text='{escaped_content}'
|
||||
WHERE cid={cid}
|
||||
"""
|
||||
cursor.execute(update_sql)
|
||||
print(f"[INFO] 更新文章成功: title={title}, cid={cid}, category={category}")
|
||||
|
||||
else:
|
||||
# ========== 执行插入逻辑 ==========
|
||||
insert_sql = (
|
||||
"INSERT INTO %s "
|
||||
f"INSERT INTO {self.__contents_table_name} "
|
||||
"(`title`, `slug`, `created`, `modified`, `text`, `order`, `authorId`, `template`, `type`, `status`, `password`, `commentsNum`, `allowComment`, `allowPing`, `allowFeed`, `parent`) "
|
||||
"VALUES "
|
||||
"('%s', NULL, %d, %d, '%s', 0, 1, NULL, 'post', 'publish', NULL, 0, '1', '1', '1', 0)"
|
||||
) % (
|
||||
self.__contents_table_name,
|
||||
escape_string(title),
|
||||
now_time_int,
|
||||
now_time_int,
|
||||
escape_string(content)
|
||||
f"VALUES ('{escape_string(title)}', NULL, {now_time_int}, {now_time_int}, '{escaped_content}', 0, 1, NULL, 'post', 'publish', NULL, 0, '1', '1', '1', 0)"
|
||||
)
|
||||
cursor.execute(insert_sql)
|
||||
cid = cursor.lastrowid
|
||||
|
||||
# slug = cid
|
||||
update_slug_sql = (
|
||||
"UPDATE %s SET slug=%d WHERE cid=%d"
|
||||
) % (self.__contents_table_name, cid, cid)
|
||||
update_slug_sql = f"UPDATE {self.__contents_table_name} SET slug={cid} WHERE cid={cid}"
|
||||
cursor.execute(update_slug_sql)
|
||||
|
||||
# 建立文章与分类的关系
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user