5 Commits

Author SHA1 Message Date
dependabot[bot]
864c197dd8 Bump minimist from 1.2.5 to 1.2.6 in /tubio-frontend-nuxt-app
Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6.
- [Release notes](https://github.com/substack/minimist/releases)
- [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6)

---
updated-dependencies:
- dependency-name: minimist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-29 10:01:26 +00:00
Leonetienne
46f32badc6 Fix really bad security vulnerability. 2022-03-24 19:48:51 +01:00
Leonetienne
fc0ae6fb40 Cmakeified 2022-03-24 17:04:28 +01:00
Leon Etienne
b0d1f1f176 Merge pull request #18 from Leonetienne/develop
Updated readme to fit the new downloader
2022-02-06 14:16:44 +00:00
Leonetienne
b2a673f887 Updated readme to fit the new downloader 2022-02-06 15:16:06 +01:00
15 changed files with 322 additions and 46 deletions

2
.gitignore vendored
View File

@@ -11,6 +11,7 @@
*.sln.docstates *.sln.docstates
# Tubio files # Tubio files
/Tubio/Tubio
config.json config.json
log.txt log.txt
log.json log.json
@@ -28,6 +29,7 @@ ffprobe.exe
mono_crash.* mono_crash.*
# Build results # Build results
build/
[Dd]ebug/ [Dd]ebug/
[Dd]ebugPublic/ [Dd]ebugPublic/
[Rr]elease/ [Rr]elease/

View File

@@ -6,7 +6,7 @@ The free, open source video downloader!
No longer do you have to rely on shady websites, shoving tons of ads in your face to fulfil your downloady needs. No longer will you be held back by artificially crippled download speeds, login-/paywalls or even watermarks. No longer do you have to rely on shady websites, shoving tons of ads in your face to fulfil your downloady needs. No longer will you be held back by artificially crippled download speeds, login-/paywalls or even watermarks.
## But what is Tubio? ## But what is Tubio?
Tubio in of itself is not a downloader, but a GUI for the widely known, open-source, public-domain cli [youtube-dl](http://youtube-dl.org/). <sup>Thanks for this awesome tool! You guys are heroes!</sup> Tubio in of itself is not a downloader, but a GUI for the widely known, open-source, public-domain cli [yt-dlp](https://github.com/yt-dlp/yt-dlp/). <sup>Thanks for this awesome tool! You guys are heroes! (Obviously, same goes for youtube-dl, which yt-dlp is based on)</sup>
The goal of Tubio is to make this awesome software more accessible. Not everyone knows how to use the command line! The goal of Tubio is to make this awesome software more accessible. Not everyone knows how to use the command line!
## But, how does it work? ## But, how does it work?
@@ -88,7 +88,7 @@ Tubio does NOT manage sessions or accounts! Everyone using your Tubio instance w
If you opt for unleashing Tubio on your LAN, i would **strongly** recommend enabling the whitelist! You can do this either in the `config.json` or in /settings. Either way, it is a json-array of strings which represent IPv4 addresses. If you opt for unleashing Tubio on your LAN, i would **strongly** recommend enabling the whitelist! You can do this either in the `config.json` or in /settings. Either way, it is a json-array of strings which represent IPv4 addresses.
## Setup (Windows) ## Setup (Windows)
#### Install youtube-dl.exe: #### Install yt-dlp.exe:
1) Download the latest Tubio build from [here](https://github.com/Leonetienne/Tubio/releases). 1) Download the latest Tubio build from [here](https://github.com/Leonetienne/Tubio/releases).
2) Create some folder on your pc. This will be the installation folder. 2) Create some folder on your pc. This will be the installation folder.
3) Dump in the contents of the Tubio build you just downloaded. 3) Dump in the contents of the Tubio build you just downloaded.
@@ -104,32 +104,35 @@ If you opt for unleashing Tubio on your LAN, i would **strongly** recommend enab
This happens as there often is some post-processing to do after downloading. This happens as there often is some post-processing to do after downloading.
### My downloads fail! ### My downloads fail!
First thing to do: Navigate to /settings and click "Update ytdl". This will update your local instance of [youtube-dl](http://youtube-dl.org/). Check the logs to see if it worked. If not, restart Tubio and try again. First thing to do: Navigate to /settings and click "Update ytdl". This will update your local instance of [yt-dlp](https://github.com/yt-dlp/yt-dlp/). Check the logs to see if it worked. If not, you can always download it yourself (from the [releases-page](https://github.com/yt-dlp/yt-dlp/releases) and put yt-dlp.exe in the same directory tubio.exe lies in.).
If it\`s still not working, you are most likely trying to download a video from a playlist with a weird url. Try a different one (The one from the share button, the one from right-clicking the video or the one from the url-bar in your browser). If it\`s still not working, you are most likely trying to download a video from a playlist with a weird url. Try a different one (The one from the share button, the one from right-clicking the video or the one from the url-bar in your browser).
If it\`s still not working, you\`re out of luck. :( If it\`s still not working, you\`re out of luck. :(
You may want to check youtube-dl compatible sites.
### I locked myself out by enabling localhost only on another device! ### I locked myself out by enabling localhost only on another device!
This can only be undone from the host. Open Tubio via, and this is important, either `localhost` or `127.0.0.1` and untick it again. If you can only ssh into the host, you can edit the `config.json` itself and restart Tubio. This can only be undone from localhost. Open Tubio via, and this is important, either `localhost` or `127.0.0.1` and untick it again. If you can only ssh into the host, you can edit the `config.json` itself and restart Tubio.
### Does it work on linux? ### Does it work on linux?
Well, technically, yes. You would have to install youtube-dl and the ffmpeg suite yourself, and compile Tubio yourself though. I have not tried it yet, but it should work. Well, technically, yes. You would have to install youtube-dl and the ffmpeg suite yourself, and compile Tubio yourself though. I have not tried it yet, but it should work.
You may get warnings for missing .exe files, which are obviously not needed on linux.
### Can i use it on my phone? ### Can I use it on my phone?
Sure. Read [this](#user-content-can-i-use-tubio-on-multiple-devices). Sure. Read [this](#user-content-can-i-use-tubio-on-multiple-devices).
### Can i use this to host my own downloader website? ### Can i use this to host my own downloader website?
On your own risk! Tubio is NOT designed for this! Also do note that tubio does NOT manage sessions or accounts! Everyone accessing this instance can see everyones downloads and access the admin panel! Tubio is really designed for one user! On your own risk! Tubio is NOT designed for this! Also do note that tubio does NOT manage sessions or accounts! Everyone accessing this instance can see everyones downloads and access the admin panel! Tubio is really designed for one user!
### XY is not working, you have to fix it now! ### XY is not working, you have to fix it now!
I do not. Tubio is a tool i originally made for myself only. I just thought it\`s nice and worth sharing. I will address issues when i have time. Feel free to submit issues and i will have a look when i get to it. :) I do not. Tubio is a tool I originally made for myself only. I just thought it\`s nice and worth sharing. I will address issues when I have time. Feel free to submit issues and I will have a look when I get to it. :)
### XY is not polished enough! ### XY is not polished enough!
This is an alpha-version. What did you expect? :D This is an alpha-version. What did you expect? :D
### Can you please add support for website XY? ### Can you please add support for website XY?
Please address the awesome team at youtube-dl, as they made the downloading-end: [github.com/ytdl-org/youtube-dl/issues](https://github.com/ytdl-org/youtube-dl/issues) Please address the awesome team at youtube-dl, as they made the downloading-end: [github.com/ytdl-org/youtube-dl/issues](https://github.com/ytdl-org/youtube-dl/issues).
The downloader tubio uses ([yt-dlp](https://github.com/yt-dlp/yt-dlp/)) is based on youtube-dl.
## Important notice! ## Important notice!
I do NOT endorse illegal downloads in any way, shape, or form. Tubio is a tool to download media from legal sources! Use Tubio at your own discretion! Neither do i provide ANY warranty in ANY way, shape, or form! I do NOT endorse illegal downloads in any way, shape, or form. Tubio is a tool to download media from legal sources! Use Tubio at your own discretion! Neither do i provide ANY warranty in ANY way, shape, or form!

51
Tubio/CMakeLists.txt Executable file
View File

@@ -0,0 +1,51 @@
cmake_minimum_required(VERSION 3.16)
project(Tubio)
# Set C++ standard
set(CMAKE_CXX_STANDARD 17)
# Add external-directories dir to include dir list
include_directories(./external_dependencies/)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17")
# Add absolutely kawaii sources to Tubio <3
add_executable(Tubio
main.cpp
ConsoleManager.cpp
ConsoleManager.h
DownloadManager.cpp
DownloadManager.h
FileSystem.cpp
FileSystem.h
Framework.cpp
Framework.h
HttpServer.cpp
HttpServer.h
Idler.cpp
Idler.h
LogHistory.cpp
LogHistory.h
LogTypes.h
Logger.cpp
Logger.h
RestQueryHandler.cpp
RestQueryHandler.h
RestResponseTemplates.cpp
RestResponseTemplates.h
TimeUnits.h
Updater.cpp
Updater.h
Version.h
XGConfig.cpp
XGConfig.h
XGControl.cpp
XGControl.h
external_dependencies/casenta/mongoose/mongoose.c
external_dependencies/casenta/mongoose/mongoose.h
external_dependencies/leonetienne/JasonPP/JasonPP.cpp
external_dependencies/leonetienne/JasonPP/JasonPP.hpp
external_dependencies/leonetienne/stringtools/StringTools.cpp
external_dependencies/leonetienne/stringtools/StringTools.h
)

View File

@@ -1,4 +1,4 @@
#include "Filesystem.h" #include "FileSystem.h"
#ifdef _WIN #ifdef _WIN
#include <Windows.h> #include <Windows.h>
#endif #endif

View File

@@ -1,4 +1,5 @@
#include "HttpServer.h" #include "HttpServer.h"
#include "external_dependencies/leonetienne/stringtools/StringTools.h"
using namespace Logging; using namespace Logging;
using namespace Rest; using namespace Rest;
@@ -137,6 +138,15 @@ void HttpServer::EventHandler(mg_connection* pNc, int ev, void* p)
return; return;
} }
std::string HttpServer::SanitizeString(std::string in) {
in = StringTools::Replace(in, '`', "\\\\`");
in = StringTools::Replace(in, '|', "\\\\|");
in = StringTools::Replace(in, '$', "\\\\$");
in = StringTools::Replace(in, "&&", "\\\\&\\\\&");
return in;
}
void HttpServer::ProcessAPIRequest(mg_connection* pNc, int ev, void* p, std::string peerAddress) void HttpServer::ProcessAPIRequest(mg_connection* pNc, int ev, void* p, std::string peerAddress)
{ {
// Get struct with http message informations // Get struct with http message informations
@@ -145,6 +155,9 @@ void HttpServer::ProcessAPIRequest(mg_connection* pNc, int ev, void* p, std::str
// Get the transmitted message body // Get the transmitted message body
std::string requestBodyRaw = FixUnterminatedString(hpm->body.p, hpm->body.len); std::string requestBodyRaw = FixUnterminatedString(hpm->body.p, hpm->body.len);
// Sanitize it
requestBodyRaw = SanitizeString(requestBodyRaw);
// Check for the body being valid json // Check for the body being valid json
if (IsJsonValid(requestBodyRaw)) if (IsJsonValid(requestBodyRaw))
{ {

View File

@@ -31,6 +31,8 @@ namespace Rest
static bool IsConnectionAllowed(std::string peer_address, std::string& denialReason); static bool IsConnectionAllowed(std::string peer_address, std::string& denialReason);
//! Will remove all `, | and && from a string to prevent remote code execution
static std::string SanitizeString(std::string in);
struct mg_mgr* pMgr; struct mg_mgr* pMgr;
struct mg_connection* pNc; struct mg_connection* pNc;

View File

@@ -1,2 +1,2 @@
#pragma once #pragma once
#define TUBIO_SERVER_VERSION (0.6) #define TUBIO_SERVER_VERSION (0.65)

View File

@@ -1,6 +1,6 @@
#pragma once #pragma once
#include <vector> #include <vector>
#include "Filesystem.h" #include "FileSystem.h"
#include "external_dependencies/leonetienne/JasonPP/JasonPP.hpp" #include "external_dependencies/leonetienne/JasonPP/JasonPP.hpp"
#include "Logger.h" #include "Logger.h"

View File

@@ -0,0 +1,155 @@
#include "StringTools.h"
#include <sstream>
std::string StringTools::Replace(const std::string& str, const char find, const std::string& subst) {
std::stringstream ss;
for (std::size_t i = 0; i < str.length(); i++)
{
if (str[i] != find)
ss << str[i];
else
ss << subst;
}
return ss.str();
}
std::string StringTools::Replace(const std::string& str, const std::string& find, const std::string& subst) {
if (find.length() == 0)
return str;
std::stringstream ss;
std::size_t posFound = 0;
std::size_t lastFound = 0;
while (posFound != std::string::npos)
{
lastFound = posFound;
posFound = str.find(find, posFound);
if (posFound != std::string::npos)
{
ss << str.substr(lastFound, posFound - lastFound) << subst;
posFound += find.length();
}
else
{
ss << str.substr(lastFound, (str.length()) - lastFound);
}
}
return ss.str();
}
std::string StringTools::Replace(const std::string& str, const char find, const char subst) {
std::stringstream ss;
ss << subst;
return Replace(str, find, ss.str());
}
std::string StringTools::Replace(const std::string& str, const std::string& find, const char subst) {
std::stringstream ss;
ss << subst;
return Replace(str, find, ss.str());
}
std::string StringTools::Lower(const std::string& str) {
std::stringstream ss;
for (std::size_t i = 0; i < str.size(); i++)
{
const char c = str[i];
// Quick-accept: regular letters
if ((c >= 'A') && (c <= 'Z'))
ss << (char)(c | 32);
// Else: keep the character as is
else ss << c;
}
return ss.str();
}
std::string StringTools::Upper(const std::string& str) {
std::stringstream ss;
for (std::size_t i = 0; i < str.size(); i++)
{
const char c = str[i];
// Quick-accept: regular letters
if ((c >= 'a') && (c <= 'z'))
ss << (char)(c & ~32);
// Else: keep the character as is
else ss << c;
}
return ss.str();
}
std::vector<std::string> StringTools::Split(const std::string& str, const std::string& seperator) {
std::vector<std::string> toRet;
// Quick-accept: str length is 0
if (str.length() == 0)
toRet.push_back("");
// Quick-accept: seperator length is 0
else if (seperator.length() == 0) {
for (const char c : str)
toRet.push_back(std::string(&c, (&c) + 1));
}
else {
std::size_t idx = 0;
while (idx != std::string::npos) {
std::size_t lastIdx = idx;
idx = str.find(seperator, idx);
// Grab our substring until the next finding of sep
if (idx != std::string::npos) {
toRet.push_back(str.substr(
lastIdx,
idx - lastIdx
));
idx += seperator.length();
}
// No more seperator found. Grab the rest until the end of the string
else {
toRet.push_back(str.substr(
lastIdx
));
}
}
}
return toRet;
}
std::string StringTools::PadLeft(const std::string& str, const char pad, const std::size_t len) {
std::stringstream ss;
for (std::size_t i = str.length(); i < len; i++)
ss << pad;
ss << str;
return ss.str();
}
std::string StringTools::PadRight(const std::string& str, const char pad, const std::size_t len) {
std::stringstream ss;
ss << str;
for (std::size_t i = str.length(); i < len; i++)
ss << pad;
return ss.str();
}

View File

@@ -0,0 +1,43 @@
#ifndef STRINGTOOLS_STRINGTOOLS_H
#define STRINGTOOLS_STRINGTOOLS_H
#include <string>
#include <vector>
/* Handy utensils to manipulate strings */
class StringTools
{
public:
//! Will replace every occurence of `find` in `str` by `subst`.
static std::string Replace(const std::string& str, const char find, const std::string& subst);
//! Will replace every occurence of `find` in `str` by `subst`.
static std::string Replace(const std::string& str, const std::string& find, const std::string& subst);
//! Will replace every occurence of `find` in `str` by `subst`.
static std::string Replace(const std::string& str, const char find, const char subst);
//! Will replace every occurence of `find` in `str` by `subst`.
static std::string Replace(const std::string& str, const std::string& find, const char subst);
//! Will make a string all-lowercase.
static std::string Lower(const std::string& str);
//! Will make a string all-uppercase.
static std::string Upper(const std::string& str);
//! Will split a string by a string seperator
static std::vector<std::string> Split(const std::string& str, const std::string& seperator);
//! Will pad a string to the left to length l
static std::string PadLeft(const std::string& str, const char pad, const std::size_t len);
//! Will pad a string to the right to length l
static std::string PadRight(const std::string& str, const char pad, const std::size_t len);
private:
// No instanciation! >:(
StringTools();
};
#endif //STRINGTOOLS_STRINGTOOLS_H

View File

@@ -1,7 +1,45 @@
#include "Framework.h" #include "Framework.h"
#ifndef _WIN
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/stat.h>
void Deamonize() {
// Fork me, kernel-san :o
pid_t fork_res = fork();
// Error handling...
if (fork_res < 0) {
std::cerr << "Aww shit! Deamonizing failed! Couldn't get forked..." << std::endl;
exit(-1);
}
// Close launcher process...
if (fork_res > 0) {
std::cout << "Successfully spawned tubio daemon... Exiting launcher gracefully..." << std::endl;
exit(0);
}
// And set the daemon process free
if (setsid() < 0) {
std::cerr << "Aww shit! Deamonizing failed! Couldn't create new session..." << std::endl;
exit(-1);
}
}
#endif
int main() int main()
{ {
#ifndef _WIN
// Deamonize();
#endif
Framework framework; Framework framework;
framework.Run(); framework.Run();

Binary file not shown.

View File

@@ -1,31 +0,0 @@
#!/bin/bash
g++ \
\
-std=c++17 \
\
-D __linux__ \
-D JASONPP_RENDER_SORTED \
\
../Tubio/main.cpp \
../Tubio/ConsoleManager.cpp \
../Tubio/DownloadManager.cpp \
../Tubio/FileSystem.cpp \
../Tubio/Framework.cpp \
../Tubio/HttpServer.cpp \
../Tubio/Logger.cpp \
../Tubio/LogHistory.cpp \
../Tubio/RestQueryHandler.cpp \
../Tubio/RestResponseTemplates.cpp \
../Tubio/XGConfig.cpp \
../Tubio/XGControl.cpp \
../Tubio/Updater.cpp \
../Tubio/Idler.cpp \
\
\
../Tubio/external_dependencies/casenta/mongoose/mongoose.c \
../Tubio/external_dependencies/leonetienne/JasonPP/JasonPP.cpp \
\
-lpthread \
\
-Wall \
-o ./tubio.out

Binary file not shown.

View File

@@ -6297,9 +6297,9 @@ minimist-options@4.1.0:
kind-of "^6.0.3" kind-of "^6.0.3"
minimist@^1.1.1, minimist@^1.1.3, minimist@^1.2.0, minimist@^1.2.5: minimist@^1.1.1, minimist@^1.1.3, minimist@^1.2.0, minimist@^1.2.5:
version "1.2.5" version "1.2.6"
resolved "https://registry.yarnpkg.com/minimist/-/minimist-1.2.5.tgz#67d66014b66a6a8aaa0c083c5fd58df4e4e97602" resolved "https://registry.yarnpkg.com/minimist/-/minimist-1.2.6.tgz#8637a5b759ea0d6e98702cfb3a9283323c93af44"
integrity sha512-FM9nNUYrRBAELZQT3xeZQ7fmMOBg6nWNmJKTcgsJeaLstP/UODVpGsr5OhXhhXg6f+qtJ8uiZ+PUxkDWcgIXLw== integrity sha512-Jsjnk4bw3YJqYzbdyBiNsPWHPfO++UGG749Cxs6peCu5Xg4nrena6OVxOYxrQTqww0Jmwt+Ref8rggumkTLz9Q==
minipass-collect@^1.0.2: minipass-collect@^1.0.2:
version "1.0.2" version "1.0.2"