convex-schema-parser (empty) → 0.1.3.0
raw patch · 21 files changed
+4086/−0 lines, 21 filesdep +HUnitdep +aesondep +base
Dependencies added: HUnit, aeson, base, containers, convex-schema-parser, deepseq, directory, filepath, fsnotify, mtl, optparse-applicative, parsec, process, split, stm, yaml
Files
- CHANGELOG.md +5/−0
- README.md +331/−0
- app/Config.hs +107/−0
- app/Dev.hs +194/−0
- app/Init.hs +79/−0
- app/Main.hs +154/−0
- convex-schema-parser.cabal +85/−0
- src/Backend/Python.hs +326/−0
- src/Backend/Python/Validator.hs +139/−0
- src/Backend/Rust.hs +1060/−0
- src/Backend/Rust/Validator.hs +107/−0
- src/Convex/Action/Parser.hs +244/−0
- src/Convex/Parser.hs +146/−0
- src/Convex/Schema/Parser.hs +282/−0
- src/Convex/Validator.hs +22/−0
- src/PathTree.hs +27/−0
- test/ActionParserTest.hs +208/−0
- test/ApiParserTest.hs +48/−0
- test/Main.hs +25/−0
- test/RustSerializationTest.hs +195/−0
- test/SchemaParserTest.hs +302/−0
+ CHANGELOG.md view
@@ -0,0 +1,5 @@+# Revision history for convex-schema-parser++## 0.1.0 -- 2025-06-30++* First version
+ README.md view
@@ -0,0 +1,331 @@+# Convex Schema Parser & Client Generator++`convex-schema-parser` is a simple command-line tool designed to parse your Convex project's schema and function definitions, generating strongly-typed API clients for both Rust and Python.++It offers two primary modes of operation:++1. A one-shot `generate` command for manual client generation.++2. A persistent `dev` mode that watches your Convex project for changes and automatically regenerates your clients, providing a seamless development experience.++> [!IMPORTANT]+> At the bottom you will find a USAGE section++## Installation++The easiest way to use `convex-schema-parser` is currently through the Cabal package manager.+We provide prebuilt binaries for `linux` & `macOS` that you can download and run directly, but `macOS` users have to allow the binary to run first since we do not sign it (yet).+A `npm` package `@parsonosai/convex-schema-parser` is also on its way and supported as soon as we get code-signing ready, we currently use a placeholder.++Installing `cabal` & `ghc` is best done using [`ghcup`](https://www.haskell.org/ghcup/). As soon as it is installed:++```bash+ghcup install ghc 9.10.1+ghcup install cabal 3.10.3.0+```++You can then build/run the tool from source.++```bash+cabal update+cabal install convex-schema-parser # If you have $HOME/.cabal/bin in your PATH.+cabal run convex-schema-parser -- --help # If you do not want to install it globally and just run it.+```++## Prerequisites++> [!NOTE]+> Everything here is also explained when you issue `conves-schema-parser init`.++Before using the tool, please ensure your environment meets the following requirements:++### 1. `pnpm` or `npm` Installation++The tool shells out to your package manager to generate TypeScript declaration files (`.d.ts`). You must have `pnpm` or `npm` installed and available in your system's PATH.++### 2. `package.json` Script++Your Convex project's `package.json` must contain a script named `declarations`. This script is responsible for running all the necessary steps to generate the `.d.ts` files for the parser to read. This often involves cleaning old artifacts, running the TypeScript compiler, and copying over pre-generated files.+Everything here will also be explained when you issue `convex-schema-parser init`, this command will also create a template `convex-parser.yaml` file.++**Example `package.json`:**++```json+{+ "scripts": {+ "declarations:clean": "rm -rf tmp",+ "declarations:build": "tsc -p tsconfig.declarations.json",+ "declarations:copy-generated": "cp -r convex/_generated tmp/declarations/",+ "declarations": "npm run declarations:clean && npm run declarations:build && npm run declarations:copy-generated",+ "test": "echo \"Error: no test specified\" && exit 1"+ }+}+```++### 3. tsconfig.declarations.json++The tool assumes a specific `tsconfig.json` file exists to guide the declaration generation process. This file should be configured to only emit declaration files into a designated output directory (e.g., `tmp/declarations`) from your source `convex/` directory.++Example `tsconfig.declarations.json`:++```json+{+ "compilerOptions": {+ // --- Output Configuration ---+ "outDir": "./tmp/declarations", // Output all files to a temporary directory+ "rootDir": "./convex", // The root of the source files we care about++ // --- Generation Settings ---+ "declaration": true, // Generate .d.ts files+ "emitDeclarationOnly": true, // Don't generate any .js files+ "skipLibCheck": true, // Speeds up compilation by not checking library files++ // --- Module Settings (to match Convex) ---+ "module": "commonjs",+ "target": "es2020"+ },+ // Tell tsc where to find the files to compile+ "include": ["convex/**/*.ts"]+}+```++# Usage++The tool is run from the command line and has two main subcommands: generate and dev.++## `generate` Command (One-Shot)++This command parses your project once, generates the specified client, and then exits. It's useful for CI/CD pipelines or manual updates.++```bash+convex-schema-parser generate --schema <path> --declarations <path> --target <lang> [-o <output_file>]++# Or if you are not using the installed binary but via cabal (same for the other commands):+cabal run convex-schema-parser -- generate --schema <path> --declarations <path> --target <lang> [-o <output_file>]+```++### Arguments:++* `--schema`: (Required) Path to your main `schema.ts` file.+ +* `--declarations`: (Required) Path to the root directory where `tsc` generated the `.d.ts` files (the `outDir` from your `tsconfig`).+ +* `--target`: (Required) The target language. Can be `Python` or `Rust`.+ +* `-o, --output`: (Optional) The file to write the generated code to. If omitted, the code will be printed to standard output.++### Example:++```bash+cabal run convex-schema-parser -- generate \+ --schema /path/to/my-project/convex/schema.ts \+ --declarations /path/to/my-project/tmp/declarations \+ --target Rust \+ --output /path/to/my-rust-app/src/convex_api.rs+```++## `dev` Command (Watch Mode)++This is the recommended mode for local development. It starts a persistent process that watches your `convex/` directory for any file changes. When a change is detected, it automatically runs the `pnpm declarations` (or `npm run declarations`) script and regenerates all clients defined in your configuration file.++```bash+convex-schema-parser dev [--config <path>]+```++### Arguments:++* `--config`: (Optional) The path to your YAML configuration file. Defaults to `convex-parser.yaml` in the current working directory.++# Configuration (`convex-parser.yaml`)++The `dev` mode is driven by a YAML configuration file. This file allows you to define multiple generation targets, enabling you to generate clients for different languages and output them to multiple locations simultaneously.++### Example `convex-parser.yaml`:++```yaml+# Configuration for the Convex Client Generator.++# (Required) The absolute path to the root of your Convex project.+# This is the directory that contains the `convex/` folder and `package.json`.+project_path: /path/to/your/convex/project++# (Required) The absolute path to the generated TypeScript declarations, relative to `project_path`.+declarations_dir: /path/to/your/tmp/declarations++# (Optional) The path where validation sandbox projects will be created.+# Defaults to `~/.config/convex-schema-parser` if omitted.+validation_path: "~/.config/convex-schema-parser"++# (Required) A list of generation targets. You can have one or more.+targets:+ # Example 1: Generate a Rust client for a backend service.+ - lang: Rust+ # A list of one or more output files for this target.+ output:+ - ../my-rust-app/src/convex_api.rs+ - ../my-other-app/src/convex_api.rs++ # Example 2: Generate a Python client for data scripts.+ - lang: Python+ output:+ - ../scripts/lib/convex_client.py+```++### Configuration Schema:++* `project_path`: The absolute path to your Convex project root directory. This directory should contain the `convex/` folder and a `package.json` file.+* `declarations_dir`: The absolute path to the directory where your TypeScript declaration files (`.d.ts`) are generated. This should be the output directory specified in your `tsconfig.declarations.json`.+* `validation_path`: (Optional) The path where validation sandbox projects will be created. If omitted, defaults to `~/.config/convex-schema-parser`.+* `targets`: A opt-level key holding a list `[]` of target configurations.+* `lang`: The target language for the client. **Must** be `Python` or `Rust`.+* `output`: A list `[]` of file paths where the generated client code will be written. Each target can have multiple output paths.++# API Usage Examples++Once you have generated your client code, you can use it in your projects.++## Python Client Example++The generated Python client uses nested classes to mirror your Convex project's file structure.++### Queries, Mutations and Actions++```python+import os+from convex import ConvexClient+# Import the generated API module (e.g., convex_api.py)+import convex_api++# 1. Instantiate the official ConvexClient with your deployment URL.+deployment_url = os.environ.get("CONVEX_URL")+client = ConvexClient(deployment_url)++# 2. Instantiate your generated API, wrapping the client.+auth_key = get_auth_key() # Replace with your method to get the auth/api/jwt key if required+client.set_auth(auth_key)++api = convex_api.API(client)++# 3. Call functions using the nested structure.+# This corresponds to the function `getProject` in `convex/functions/projects.ts`.+# The generated API reraises any exceptions from the Convex client, so you can handle them as needed.+# Additionally, we use `pydantic` for type validation, so we raise these exceptions as well.+try:+ project_id = convex_api.Id("prj_...")+ project = api.functions.projects.get_project(project_id)+ if project:+ print(f"Successfully fetched project: {project.project_name}")+ else:+ print("Project not found.")+except Exception as e:+ print(f"An error occurred: {e}")+```++### Subscriptions++```python+from convex import ConvexError++# ... (assuming `api` is already instantiated and authenticated)++try:+ # 1. Call the generated `subscribe_*` method. This returns a generator instantly.+ tenant_id = convex_api.Id("tnt_...")+ project_subscription = api.functions.queries.subscribe_fetch_projects(tenant_id)++ print("Subscribed to projects. Waiting for updates... (Press Ctrl+C to stop)")++ # 2. The `for` loop starts the subscription and blocks until the first value+ # is received. The loop body will run again for each subsequent update.+ for updated_projects in project_subscription:+ # 3. `updated_projects` is already a fully validated Pydantic model+ # (e.g., list[FetchProjectsReturnObject]).+ print(f"Received update with {len(updated_projects)} projects:")+ for project in updated_projects:+ print(f" - ID: {project._id}, Name: {project.project_name}")++except ConvexError as e:+ print(f"Subscription failed with an error: {e}")+except KeyboardInterrupt:+ print("\nSubscription stopped by user.")+```+++## Rust Client Example++The generated Rust client uses a method-based API which works with Rust's ownership and borrowing rules.++### Queries, Mutations and Actions++```rust+// Assuming the generated module is named `convex_api`.+use convex_api::{Api, Id, types::ProjectsDoc};+use convex::ConvexClient;+use anyhow::Result;++#[tokio::main]+async fn main() -> anyhow::Result<()> {+ // 1. Instantiate and authenticate the official ConvexClient.+ let convex_url = std::env::var("CONVEX_URL")?;+ let auth_key = get_auth_key(); // Replace with your method to get the auth/api/jwt key if required+ + let mut convex_client = ConvexClient::new(&convex_url).await?;+ convex_client.set_auth(&auth_key);++ // 2. Instantiate your generated API, wrapping the client.+ let mut api = Api::new(convex_client);++ // 3. Call functions using the nested, method-chaining API.+ let project_id = Id::<ProjectsDoc>::new("prj_...".to_string());+ + // This corresponds to the function `getProject` in `convex/functions/projects.ts`.+ match api.functions().projects().get_project(&project_id).await {+ Ok(Some(project)) => {+ println!("Successfully fetched project: {}", project.project_name.unwrap_or_default());+ }+ Ok(None) => {+ println!("Project not found.");+ }+ Err(e) => {+ eprintln!("An error occurred: {}", e);+ }+ }++ Ok(())+}+```++### Subscriptions++```rust+use futures_util::stream::StreamExt;++// ... (assuming `api` is already instantiated and authenticated)++async fn run_subscription() -> anyhow::Result<()> {+ // 1. Call the generated `subscribe_*` method.+ let tenant_id = convex_api::Id::<convex_api::types::TenantsDoc>::new("tnt_...".to_string());+ let mut project_subscription = api.functions().queries().subscribe_fetch_projects(&tenant_id).await?;++ println!("Subscribed to projects. Waiting for updates... (Press Ctrl+C to stop)");++ // 2. The `while let` loop asynchronously polls the stream for new items.+ while let Some(result) = project_subscription.next().await {+ // 3. Each `result` is a `Result<T, ApiError>`, where T is your strongly-typed+ // return value (e.g., Vec<FetchProjectsReturnObject>).+ match result {+ Ok(updated_projects) => {+ println!("Received update with {} projects:", updated_projects.len());+ for project in updated_projects {+ println!(" - ID: {}, Name: {}", project._id, project.project_name.unwrap_or_default());+ }+ }+ Err(e) => {+ eprintln!("Received an error in the subscription stream: {}", e);+ }+ }+ }++ Ok(())+}+```
+ app/Config.hs view
@@ -0,0 +1,107 @@+{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE OverloadedStrings #-}++module Config+ ( loadConfig,+ Config (..),+ TargetConfig (..),+ GenTarget (..),+ )+where++import Data.Aeson+import Data.Char (isAlphaNum)+import Data.List (isPrefixOf)+import Data.Maybe (fromMaybe)+import Data.Traversable (forM)+import Data.Yaml (ParseException, decodeFileEither)+import GHC.Generics (Generic)+import System.Directory (getHomeDirectory)+import System.Environment (getEnv, lookupEnv)+import System.FilePath (joinPath, splitPath, (</>))++data GenTarget = Python | Rust+ deriving (Show, Eq, Enum, Bounded, Generic)++-- Make GenTarget parseable from YAML (as a string)+instance FromJSON GenTarget where+ parseJSON = withText "GenTarget" $ \t ->+ case t of+ "Python" -> pure Python+ "Rust" -> pure Rust+ _ -> fail "Unknown target language. Use 'Python' or 'Rust'."++data TargetConfig = TargetConfig+ { lang :: GenTarget,+ output :: [FilePath]+ }+ deriving (Show, Generic)++instance FromJSON TargetConfig++data ConfigRead = ConfigRead+ { project_path :: FilePath,+ declarations_dir :: FilePath,+ targets :: [TargetConfig],+ validation_path :: Maybe (FilePath)+ }+ deriving (Show, Generic)++instance FromJSON ConfigRead++data Config = Config+ { projectPath :: FilePath,+ declarationsDir :: FilePath,+ targetList :: [TargetConfig],+ validationPath :: FilePath+ }+ deriving (Show, Generic)++-- | Loads and parses the YAML configuration file.+loadConfig :: FilePath -> IO (Either ParseException Config)+loadConfig p = do+ r <- decodeFileEither p+ r' <- case r of+ Right config -> do+ cfg <-+ Config+ <$> (expandFullPath $ project_path config)+ <*> (expandFullPath $ declarations_dir config)+ <*> (mapM expandInTargetConfig (targets config))+ <*> (expandFullPath $ maybe "~/.config/convex-schema-parser" id (validation_path config))+ return $ Right cfg+ Left v -> return (Left v :: Either ParseException Config)+ return r'++expandInTargetConfig :: TargetConfig -> IO TargetConfig+expandInTargetConfig (TargetConfig lang outputs) = do+ expandedOutputs <- mapM expandFullPath outputs+ return $ TargetConfig lang expandedOutputs++expandEnvInPath :: FilePath -> IO FilePath+expandEnvInPath input = do+ let components = splitPath input -- preserves trailing slashes+ expanded <- forM components $ \part ->+ case part of+ ('$' : rest) -> do+ let varName = takeWhile isAlphaNum rest+ mVal <- lookupEnv varName+ return $ fromMaybe part mVal+ _ -> return part+ return $ joinPath expanded++expandUserPath :: FilePath -> IO FilePath+expandUserPath path+ | "~/" `isPrefixOf` path = do+ home <- getHomeDirectory+ return $ home </> drop 2 path+ | "$HOME/" `isPrefixOf` path = do+ home <- getEnv "HOME"+ return $ home </> drop 6 path+ | otherwise = return path++expandFullPath :: FilePath -> IO FilePath+expandFullPath path = do+ expandedEnv <- expandEnvInPath path+ expandedUser <- expandUserPath expandedEnv+ return expandedUser
+ app/Dev.hs view
@@ -0,0 +1,194 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE LambdaCase #-}+{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE ScopedTypeVariables #-}++module Dev (runDevMode, DevOptions (..)) where++import qualified Backend.Python as Python+import qualified Backend.Python.Validator as Python.Validator+import qualified Backend.Rust as Rust+import qualified Backend.Rust.Validator as Rust.Validator+import qualified Config+import Control.Concurrent (forkIO, threadDelay)+import Control.Concurrent.STM (TVar, atomically, newTVarIO, readTVar, retry, writeTVar)+import Control.DeepSeq (rnf)+import Control.Exception+import Control.Exception (evaluate)+import Control.Monad (forM_, forever, when)+import Control.Monad.IO.Class (MonadIO, liftIO)+import qualified Convex.Parser as P+import qualified Convex.Validator as Validator+import System.Directory (doesFileExist, findExecutable)+import System.Exit (ExitCode (..), exitFailure)+import System.FSNotify+import System.IO.Error (isAlreadyInUseError)+import System.Process (CreateProcess (..), StdStream (..), createProcess, proc, waitForProcess)++data DevOptions = DevOptions+ { convexProjectPath :: FilePath,+ declarationsDir :: FilePath,+ config :: Config.Config+ }++-- | Searches for an available package manager, preferring pnpm.+findPackageManager :: IO (Maybe String)+findPackageManager = do+ pnpmPath <- findExecutable "pnpm"+ case pnpmPath of+ Just _ -> return (Just "pnpm")+ Nothing -> do+ npmPath <- findExecutable "npm"+ case npmPath of+ Just _ -> return (Just "npm")+ Nothing -> return Nothing++-- | Runs the development mode file watcher with debouncing.+runDevMode :: Config.Config -> IO ()+runDevMode config = do+ -- TVar to signal that a change has occurred.+ dirtyVar <- newTVarIO True -- Start dirty to trigger an initial run.++ -- Fork a worker thread that handles the generation.+ _ <- forkIO (workerThread dirtyVar config)++ -- The main thread will watch for file changes.+ withManager $ \mgr -> do+ putStrLn $ "Watching for changes in: " ++ convexPath+ watchTree+ mgr+ convexPath+ (const True) -- Watch all events+ ( \event -> do+ putStrLn $ "[Watcher] Detected change: " ++ showEvent event+ -- Atomically set the dirty flag. This is fast and non-blocking.+ atomically $ writeTVar dirtyVar True+ )++ -- Keep the main thread alive.+ forever $ threadDelay 1000000+ where+ convexPath = Config.projectPath config ++ "/convex"+ showEvent (Added path _ _) = "Added: " ++ path+ showEvent (Modified path _ _) = "Modified: " ++ path+ showEvent (Removed path _ _) = "Removed: " ++ path+ showEvent _ = "Unknown event"++-- | The worker thread loop that performs debouncing.+workerThread :: TVar Bool -> Config.Config -> IO ()+workerThread dirtyVar config = forever $ do+ -- Wait for a signal that something has changed.+ atomically $ do+ dirty <- readTVar dirtyVar+ -- The `check` function will retry the transaction if the condition is false.+ -- This effectively makes the thread sleep until `dirty` is True.+ when (not dirty) retry++ -- Once signaled, reset the flag and wait for the debounce period.+ atomically $ writeTVar dirtyVar False+ putStrLn "[Worker] Change detected. Debouncing for 300ms..."+ threadDelay 300000 -- 300ms debounce period.++ -- After debouncing, check if another change came in. If not, generate.+ isDirty <- atomically (readTVar dirtyVar)+ if isDirty+ then putStrLn "[Worker] Further changes detected. Restarting debounce timer."+ else+ handleGenerationEvent config `catch` \(e :: SomeException) -> do+ putStrLn $ "[Worker] Error during generation: " ++ show e++-- | Handles the logic for regenerating declarations and the clients.+handleGenerationEvent :: Config.Config -> IO ()+handleGenerationEvent config = do+ findPackageManager >>= \case+ Nothing -> putStrLn "\n[ERROR] Neither 'pnpm' nor 'npm' could be found in your system's PATH.\nPlease install one of them to use the dev watcher.\n"+ Just pm -> do+ putStrLn "[Worker] Regenerating .d.ts files..."+ let pkgMgrCmd = (proc pm ["declarations"]) {cwd = Just $ Config.projectPath config}+ (_, _, _, pkgMgrHandle) <- createProcess pkgMgrCmd+ exitCode <- waitForProcess pkgMgrHandle+ case exitCode of+ ExitSuccess -> do+ putStrLn "[Worker] Declaration files regenerated successfully."+ generateAndWriteCode config+ ExitFailure code ->+ putStrLn $ "[Worker] Error: 'pnpm declarations' failed with exit code " ++ show code++-- | Generates the code and writes it to the target files if it has changed.+generateAndWriteCode :: Config.Config -> IO ()+generateAndWriteCode config = do+ putStrLn "[Worker] Parsing schema and generating clients..."+ projectResult <- P.parseProject (Config.projectPath config ++ "/convex/schema.ts") $ Config.declarationsDir config+ case projectResult of+ Left err ->+ putStrLn $ "[Worker] Error parsing project: " ++ err+ Right project ->+ -- Iterate over each target defined in the config+ forM_ (Config.targetList config) $ \Config.TargetConfig {..} -> do+ putStrLn $ "[Worker] Generating for target: " ++ show lang+ let generatedCode = case lang of+ Config.Python -> Python.generatePythonCode project+ Config.Rust -> Rust.generateRustCode project++ let validatorAction :: (MonadFail m, MonadIO m, Validator.Validator m) => m String+ validatorAction = do+ Validator.setup+ Validator.validate generatedCode >>= \case+ Just formattedAndChecked -> do+ liftIO $ putStrLn "[Worker] Rust code validation successful."+ return formattedAndChecked+ Nothing -> do+ fail "[Worker] Rust code validation failed. Check the output above."++ checkedCode <- case lang of+ Config.Rust -> do+ Rust.Validator.run (Rust.Validator.RustValidatorEnv (Config.validationPath config ++ "/rust_validation_project")) validatorAction+ Config.Python -> do+ -- Python.Validator.run (Python.Validator.PythonValidatorEnv (Config.projectPath config ++ "/python_validation_project")) validatorAction+ return generatedCode++ -- Write to each output file defined for the target+ forM_ output $ \path -> do+ !oldContentResult <- readTargetFileWithRetry path+ putStrLn $ "[Worker] Checking " ++ path ++ " for changes..."+ case oldContentResult of+ Left err -> putStrLn $ "[Worker] Error reading existing file: " ++ show err+ Right oldContent ->+ if checkedCode /= oldContent+ then do+ putStrLn $ "[Worker] -> Changes detected for " ++ path+ putStrLn "[Worker] -> Writing new content..."+ writeResult <- writeFileWithRetry path checkedCode+ case writeResult of+ Left err -> putStrLn $ "[Worker] Error writing file: " ++ show err+ Right () -> putStrLn $ "[Worker] -> Success! Updated " ++ path+ else+ putStrLn $ "[Worker] -> No changes detected for " ++ path++-- | Safely reads the content of a file, retrying on busy errors.+readTargetFileWithRetry :: FilePath -> IO (Either IOException String)+readTargetFileWithRetry path = do+ exists <- doesFileExist path+ if exists+ then tryWithRetries 10 200000 $ do+ -- 10 retries, 200ms apart+ content <- readFile path+ _ <- evaluate (rnf content) -- Force evaluation to ensure the file handle is released.+ return content+ else return (Right "")++-- | Writes content to a file, retrying on busy errors.+writeFileWithRetry :: FilePath -> String -> IO (Either IOException ())+writeFileWithRetry path content = tryWithRetries 10 200000 (writeFile path content)++-- | A helper to retry an IO action if it fails due to a "resource busy" error.+tryWithRetries :: Int -> Int -> IO a -> IO (Either IOException a)+tryWithRetries 0 _ action = try action -- Last attempt, return the result+tryWithRetries n delayMicroseconds action = do+ result <- try action+ case result of+ Left e | isAlreadyInUseError e -> do+ putStrLn $ "[Retry] File is busy, retrying in " ++ show (delayMicroseconds `div` 1000) ++ "ms..."+ threadDelay delayMicroseconds+ tryWithRetries (n - 1) delayMicroseconds action+ _ -> return result
+ app/Init.hs view
@@ -0,0 +1,79 @@+module Init (runInit) where++defaultConfig :: String+defaultConfig =+ unlines+ [ "# Configuration for the Convex Client Generator.",+ "",+ "# (Required) The absolute path to the root of your Convex project.",+ "# This is the directory that contains the `convex/` folder and `package.json`.",+ "# NOTE: The parser expands `~` and `$HOME` environment variables.",+ "project_path: \"/path/to/your/convex/project\"",+ "",+ "# (Required) The absolute path to the generated TypeScript declarations.",+ "declarations_dir: \"/path/to/your/tmp/declarations\"",+ "",+ "# (Optional) The path where validation sandbox projects will be created.",+ "# Defaults to `~/.config/convex-schema-parser` if omitted.",+ "validation_path: \"~/.config/convex-schema-parser\"",+ "",+ "# (Required) A list of generation targets. You can have one or more.",+ "targets:",+ " # Example 1: Generate a Rust client for a backend service.",+ " - lang: Rust",+ " # A list of one or more output files for this target.",+ " output:",+ " - /path/to/my-rust-app/src/convex_api.rs",+ "",+ " # Example 2: Generate a Python client for data scripts.",+ " - lang: Python",+ " output:",+ " - /path/to/scripts/lib/convex_client.py"+ ]++-- | Creates a default configuration file at the specified path.+runInit :: FilePath -> IO ()+runInit configPath = do+ putStrLn $ "Creating default configuration file at: " ++ configPath+ writeFile configPath defaultConfig+ putStrLn "Configuration file created successfully. Please edit it with your project paths."+ mapM_+ putStrLn+ [ "You will also need a `tsconfig.declarations.json` file next to your `package.json`.",+ "",+ "Example `tsconfig.declarations.json`:",+ "",+ "```",+ "{",+ " \"compilerOptions\": {",+ " // --- Output Configuration ---",+ " \"outDir\": \"./tmp/declarations\", // Output all files to a temporary directory",+ " \"rootDir\": \"./convex\", // The root of the source files we care about",+ "",+ " // --- Generation Settings ---",+ " \"declaration\": true, // Generate .d.ts files",+ " \"emitDeclarationOnly\": true, // Don't generate any .js files",+ " \"skipLibCheck\": true, // Speeds up compilation by not checking library files",+ "",+ " // --- Module Settings (to match Convex) ---",+ " \"module\": \"commonjs\",",+ " \"target\": \"es2020\"",+ " },",+ " // Tell tsc where to find the files to compile",+ " \"include\": [\"convex/**/*.ts\"]",+ "}",+ "```",+ "",+ "Also, adapt your `package.config` with a script that is called by this generator:",+ "",+ "```",+ " \"scripts\": {",+ " \"declarations:clean\": \"rm -rf tmp\",",+ " \"declarations:build\": \"tsc -p tsconfig.declarations.json\",",+ " \"declarations:copy-generated\": \"cp -r convex/_generated tmp/declarations/\",",+ " \"declarations\": \"npm run declarations:clean && npm run declarations:build && npm run declarations:copy-generated\",",+ " },",+ "```",+ "",+ "If that is done, you are all set to use `dev`. (:"+ ]
+ app/Main.hs view
@@ -0,0 +1,154 @@+module Main (main) where++import qualified Backend.Python as Python+import qualified Backend.Rust as Rust+import qualified Config+import qualified Convex.Parser as P+import Data.List (intercalate)+import Dev (runDevMode)+import qualified Dev+import qualified Init+import Options.Applicative+import System.Directory (doesFileExist)+import System.Exit (exitFailure)++data Command+ = Generate GenerateOptions+ | Dev DevCliOptions+ | Init InitOptions++data GenerateOptions = GenerateOptions+ { schemaPath :: FilePath,+ declarationsDir :: FilePath,+ target :: Config.GenTarget,+ outputFile :: Maybe FilePath+ }++data DevCliOptions = DevCliOptions+ { devCliConfigPath :: FilePath+ }++data InitOptions = InitOptions+ { initConfigPath :: FilePath+ }++-- We can reuse the GenTarget Read instance from Config.hs if it were exported,+-- but redefining it here keeps Main self-contained for parsing.+instance Read Config.GenTarget where+ readsPrec _ "Python" = [(Config.Python, "")]+ readsPrec _ "Rust" = [(Config.Rust, "")]+ readsPrec _ _ = []++availableTargets :: String+availableTargets = "Available: " ++ intercalate ", " (map show [(minBound :: Config.GenTarget) ..])++generateOptionsParser :: Parser GenerateOptions+generateOptionsParser =+ GenerateOptions+ <$> strOption+ ( long "schema"+ <> metavar "SCHEMA_PATH"+ <> help "Path to the source schema.ts file"+ )+ <*> strOption+ ( long "declarations"+ <> metavar "DECL_DIR"+ <> help "Path to the root of the tsc-generated declarations directory"+ )+ <*> option+ auto+ ( long "target"+ <> metavar "TARGET"+ <> value Config.Python+ <> showDefault+ <> help ("Target language for code generation. " ++ availableTargets)+ )+ <*> optional+ ( strOption+ ( long "output"+ <> short 'o'+ <> metavar "OUTPUT_FILE"+ <> help "File to write the generated code to (prints to stdout if omitted)"+ )+ )++devCliOptionsParser :: Parser DevCliOptions+devCliOptionsParser =+ DevCliOptions+ <$> strOption+ ( long "config"+ <> metavar "CONFIG_FILE"+ <> value "convex-parser.yaml"+ <> showDefault+ <> help "Path to the YAML config file"+ )++initOptionsParser :: Parser InitOptions+initOptionsParser =+ InitOptions+ <$> strOption+ ( long "config"+ <> metavar "CONFIG_FILE"+ <> value "convex-parser.yaml"+ <> showDefault+ <> help "Path where the default YAML config file will be created"+ )++commandParser :: Parser Command+commandParser =+ subparser+ ( command "generate" (info (Generate <$> generateOptionsParser <**> helper) (progDesc "Generate a client once and exit"))+ <> command "dev" (info (Dev <$> devCliOptionsParser <**> helper) (progDesc "Watch for changes and regenerate clients from a config file"))+ <> command "init" (info (Init <$> initOptionsParser <**> helper) (progDesc "Initialize a descriptive default config"))+ )++main :: IO ()+main = do+ cmd <- customExecParser p optsParserInfo+ case cmd of+ Generate opts -> runGenerate opts+ Dev opts -> runDev opts+ Init opts -> Init.runInit (initConfigPath opts)+ where+ optsParserInfo =+ info+ (commandParser <**> helper)+ ( fullDesc+ <> progDesc "Generate typed clients from a Convex schema and function definitions."+ <> header "convex-parser - A code generator for Convex"+ )+ p = prefs showHelpOnError++runGenerate :: GenerateOptions -> IO ()+runGenerate opts = do+ projectResult <- P.parseProject (schemaPath opts) (declarationsDir opts)+ case projectResult of+ Left err -> do+ putStrLn $ "Error parsing project: " ++ err+ exitFailure+ Right project ->+ let generatedCode = case target opts of+ Config.Python -> Python.generatePythonCode project+ Config.Rust -> Rust.generateRustCode project+ in case outputFile opts of+ Just path -> writeFile path generatedCode+ Nothing -> putStrLn generatedCode++runDev :: DevCliOptions -> IO ()+runDev opts = do+ configExists <- doesFileExist (devCliConfigPath opts)+ if not configExists+ then do+ putStrLn $ "Error: Config file not found at " ++ devCliConfigPath opts+ exitFailure+ else do+ configResult <- Config.loadConfig (devCliConfigPath opts)+ case configResult of+ Left err -> do+ putStrLn "Error parsing config file:"+ print err+ exitFailure+ Right config ->+ let declarationsDir = Config.declarationsDir config+ devOpts = Dev.DevOptions (Config.projectPath config) declarationsDir config+ in runDevMode config
+ convex-schema-parser.cabal view
@@ -0,0 +1,85 @@+cabal-version: 3.0+name: convex-schema-parser+version: 0.1.3.0+license: MIT+author: Norbert Dzikowski+maintainer: lambdax.one@icloud.com+copyright: (c) 2025 Parsonos Corporation+build-type: Simple+extra-doc-files: CHANGELOG.md, README.md++synopsis: A type-safe client generator for Convex for both Rust and Python.+description:+ A command-line tool designed to parse your Convex project's schema and+ function definitions, generating strongly-typed API clients for both Rust+ and Python.+category: Development, Code Generation, Web+homepage: https://github.com/parsonosai/convex-schema-parser+bug-reports: https://github.com/parsonosai/convex-schema-parser/issues++tested-with: GHC == 9.10.1, GHC == 9.6.7++source-repository head+ type: git+ location: https://github.com/parsonosai/convex-schema-parser.git++common warnings+ ghc-options: -Wall++library convex-schema-parser-lib+ import: warnings+ exposed-modules: Convex.Schema.Parser+ , Convex.Action.Parser+ , Convex.Parser+ , Convex.Validator+ , PathTree+ , Backend.Python+ , Backend.Python.Validator+ , Backend.Rust+ , Backend.Rust.Validator+ build-depends: base >= 4.18.3 && < 4.21+ , parsec >= 3.1.17 && < 3.2+ , mtl >= 2.3.1 && < 2.4+ , containers >= 0.7 && < 0.8+ , filepath >= 1.5.2 && < 1.6+ , split >= 0.2.5 && < 0.3+ , directory >= 1.3.8 && < 1.4+ , process >= 1.6.19 && < 1.7+ hs-source-dirs: src+ default-language: Haskell2010++executable convex-schema-parser+ main-is: Main.hs+ other-modules: Dev+ , Config+ , Init+ build-depends: base,+ convex-schema-parser-lib,+ parsec,+ optparse-applicative >= 0.19.0 && < 0.20,+ fsnotify >= 0.4.3 && < 0.5,+ process,+ directory,+ yaml >= 0.11.11 && < 0.12,+ aeson >= 2.2.3 && < 2.3,+ stm >= 2.5.3 && < 2.6,+ deepseq >= 1.4.8 && < 1.6,+ filepath+ hs-source-dirs: app+ default-language: Haskell2010++test-suite test-convex-parser+ type: exitcode-stdio-1.0+ main-is: Main.hs+ other-modules: ActionParserTest+ , ApiParserTest+ , SchemaParserTest+ , RustSerializationTest+ hs-source-dirs: test+ build-depends: base,+ HUnit >=1.6.0 && < 1.7,+ parsec,+ mtl,+ containers,+ convex-schema-parser-lib+ default-language: Haskell2010
+ src/Backend/Python.hs view
@@ -0,0 +1,326 @@+{-# LANGUAGE OverloadedStrings #-}++module Backend.Python (generatePythonCode) where++import qualified Convex.Action.Parser as Action+import qualified Convex.Parser as P+import qualified Convex.Schema.Parser as Schema+import Data.Char (isUpper, toLower, toUpper)+import Data.List (intercalate, isPrefixOf, nub)+import qualified Data.Map.Strict as Map+import Data.Maybe (catMaybes)+import PathTree++-- Helper function to prepend a given number of spaces (4 per level).+indent :: Int -> String -> String+indent n s = replicate (n * 4) ' ' ++ s++generatePythonCode :: P.ParsedProject -> String+generatePythonCode project =+ let (constantsCode, nestedModelsFromConstants) = generateAllConstants (P.ppConstants project)+ (schemaCode, nestedFromFields) = generateAllTables (P.ppSchema project)+ (apiClassCode, nestedFromFuncs) = generateApiClass (P.ppFunctions project)+ allNestedCode = unlines . nub $ nestedFromFields ++ nestedModelsFromConstants ++ nestedFromFuncs+ aliasesCode = generateAliases (P.ppSchema project)+ in unlines+ [ generateHeader,+ constantsCode,+ allNestedCode,+ schemaCode,+ aliasesCode,+ apiClassCode+ ]++-- | Generates the static header for the Python file.+generateHeader :: String+generateHeader =+ unlines+ [ "from typing import Any, Generic, Iterator, Literal, TypeVar",+ "",+ "from convex import ConvexClient",+ "from pydantic import BaseModel, Field, TypeAdapter, ValidationError",+ "from pydantic_core import core_schema",+ "",+ "",+ "T = TypeVar('T')",+ "class Id(str, Generic[T]):",+ " @classmethod",+ " def __get_pydantic_core_schema__(cls, s, h) -> core_schema.CoreSchema:",+ " return core_schema.no_info_after_validator_function(cls, core_schema.str_schema())",+ ""+ ]++-- | Generates Python type aliases for all the named constants.+generateAllConstants :: Map.Map String Schema.ConvexType -> (String, [String])+generateAllConstants constants =+ let results = map (generateConstant . fst) (Map.toList constants)+ in (unlines $ map fst results, concatMap snd results)+ where+ generateConstant name =+ let constType = constants Map.! name+ (pyType, _, _, nested) = toPythonTypeParts name constType+ in (name ++ " = " ++ pyType, nested)++-- | Generates Pydantic BaseModel classes for all tables.+generateAllTables :: Schema.Schema -> (String, [String])+generateAllTables (Schema.Schema tables) =+ let (tableCodes, nested) = unzip $ map generateTable tables+ in (unlines tableCodes, concat nested)++-- | Generates a single Pydantic BaseModel class for a table.+generateTable :: Schema.Table -> (String, [String])+generateTable table =+ let className = toClassName (Schema.tableName table)+ idField = Schema.Field "_id" (Schema.VId (Schema.tableName table))+ creationTimeField = Schema.Field "_creationTime" Schema.VNumber+ allFields = [idField, creationTimeField] ++ Schema.tableFields table+ (fieldLines, nestedModelsFromFields) = unzip $ map (generateField className) allFields+ tableCode =+ unlines+ [ "class " ++ className ++ "(BaseModel):",+ unlines fieldLines,+ "",+ indent 1 "class Config:",+ indent 2 "populate_by_name: bool = True"+ ]+ in (tableCode, concat nestedModelsFromFields)++-- | Generates singular type aliases for all table documents.+generateAliases :: Schema.Schema -> String+generateAliases (Schema.Schema tables) =+ let header = "\n# --- Singular Type Aliases for Ergonomics ---\n"+ in header ++ (unlines $ map toAlias tables)+ where+ toAlias t = toSingular (Schema.tableName t) ++ " = " ++ toClassName (Schema.tableName t)++-- | Generates the code for a single Python function wrapper.+generateFunction :: Int -> Action.ConvexFunction -> (String, [String])+generateFunction level func =+ let funcName = Action.funcName func+ (argSignature, payloadMapping, nestedFromArgs) = generateArgSignature funcName (Action.funcArgs func)+ funcNameSnake = toSnakeCase funcName+ (rawReturnHint, isModelReturn, nestedFromReturn) = getReturnType funcName (Action.funcReturn func)++ handlerCall = case Action.funcType func of+ Action.Query -> "self._client.query"+ Action.Mutation -> "self._client.mutation"+ Action.Action -> "self._client.action"++ fullFuncPath = "\"" ++ Action.funcPath func ++ ":" ++ funcName ++ "\""++ (finalReturnHint, tryBlock) = case Action.funcReturn func of+ Schema.VVoid ->+ ( "None",+ unlines+ [ indent (level + 2) (handlerCall ++ "(" ++ fullFuncPath ++ ", payload)"),+ indent (level + 2) "return"+ ]+ )+ _ ->+ let hint = rawReturnHint ++ " | None"+ rawResultDeclaration =+ if isModelReturn+ then "raw_result = "+ else "raw_result: " ++ hint ++ " = "+ validationLogic =+ if isModelReturn+ then+ if "list[" `isPrefixOf` rawReturnHint+ then "TypeAdapter(" ++ rawReturnHint ++ ").validate_python(raw_result)"+ else rawReturnHint ++ ".model_validate(raw_result)"+ else "raw_result"+ body =+ unlines+ [ indent (level + 2) (rawResultDeclaration ++ handlerCall ++ "(" ++ fullFuncPath ++ ", payload)"),+ indent (level + 2) "if raw_result is None:",+ indent (level + 3) "return None",+ indent (level + 2) ("return " ++ validationLogic)+ ]+ in (hint, body)++ funcCode =+ unlines+ [ indent level ("def " ++ funcNameSnake ++ "(self, " ++ argSignature ++ ") -> " ++ finalReturnHint ++ ":"),+ indent (level + 1) ("\"\"\"Wraps the " ++ fullFuncPath ++ " " ++ show (Action.funcType func) ++ ".\"\"\""),+ indent (level + 1) ("payload: dict[str, Any] = {" ++ payloadMapping ++ "}"),+ indent (level + 1) "try:",+ tryBlock,+ indent (level + 1) "except ValidationError as e:",+ indent (level + 2) ("print(f\"Validation error in '" ++ funcNameSnake ++ "': {e}\")"),+ indent (level + 2) "raise",+ indent (level + 1) "except Exception as e:",+ indent (level + 2) ("print(f\"Error in '" ++ funcNameSnake ++ "': {e}\")"),+ indent (level + 2) "raise"+ ]+ in (funcCode, nestedFromArgs ++ nestedFromReturn)++generateSubscriptionFunction :: Int -> Action.ConvexFunction -> (String, [String])+generateSubscriptionFunction level func =+ let funcName = Action.funcName func+ (argSignature, payloadMapping, nestedFromArgs) = generateArgSignature funcName (Action.funcArgs func)+ funcNameSnake = "subscribe_" ++ toSnakeCase funcName+ (returnHint, _, nestedFromReturn) = getReturnType funcName (Action.funcReturn func)+ finalReturnHint = "Iterator[" ++ returnHint ++ "]"+ fullFuncPath = "\"" ++ Action.funcPath func ++ ":" ++ funcName ++ "\""++ adapterCreation = indent (level + 1) ("adapter = TypeAdapter(" ++ returnHint ++ ")")+ validationLogic = indent (level + 3) "validated_result = adapter.validate_python(raw_result)"++ funcCode =+ unlines+ [ indent level ("def " ++ funcNameSnake ++ "(self, " ++ argSignature ++ ") -> " ++ finalReturnHint ++ ":"),+ indent (level + 1) ("\"\"\"Subscribes to the " ++ fullFuncPath ++ " query.\"\"\""),+ indent (level + 1) ("payload: dict[str, Any] = {" ++ payloadMapping ++ "}"),+ indent (level + 1) ("raw_subscription = self._client.subscribe(" ++ fullFuncPath ++ ", payload)"),+ adapterCreation,+ indent (level + 1) "for raw_result in raw_subscription:",+ indent (level + 2) "try:",+ validationLogic,+ indent (level + 3) "yield validated_result",+ indent (level + 2) "except ValidationError as e:",+ indent (level + 3) "print(f\"Validation error in subscription update: {e}\")",+ indent (level + 3) "continue"+ ]+ in (funcCode, nestedFromArgs ++ nestedFromReturn)++generateApiStructure :: Int -> PathTree -> ([String], [String], [String])+generateApiStructure level (DirNode dir) =+ let (inits, defs, nested) = foldl processEntry ([], [], []) (Map.toList dir)+ in (inits, defs, nested)+ where+ processEntry (is, ds, ns) (_name, FuncNode func) =+ let (funcDef, nestedFromFunc) = generateFunction level func+ (subDef, nestedFromSub) =+ if Action.funcType func == Action.Query+ then generateSubscriptionFunction level func+ else ("", [])+ in (is, ds ++ [funcDef, subDef], ns ++ nestedFromFunc ++ nestedFromSub)+ processEntry (is, ds, ns) (name, DirNode subDir) =+ let className = capitalize name+ attrName = toSnakeCase name+ initLine = "self." ++ attrName ++ " = self." ++ className ++ "(self._client)"+ (subInits, subDefs, nestedFromSub) = generateApiStructure (level + 1) (DirNode subDir)+ classDef =+ unlines $+ [ "",+ indent level ("class " ++ className ++ ":"),+ indent (level + 1) "def __init__(self, client: ConvexClient):",+ indent (level + 2) "self._client = client"+ ]+ ++ map (indent (level + 2)) subInits+ ++ subDefs+ in (is ++ [initLine], ds ++ [classDef], ns ++ nestedFromSub)+generateApiStructure _ (FuncNode _) = ([], [], [])++generateApiClass :: [Action.ConvexFunction] -> (String, [String])+generateApiClass funcs =+ let tree = buildPathTree funcs+ (initLines, definitionLines, nestedModels) = generateApiStructure 1 tree+ header =+ [ "\n# --- API Client Class ---\n",+ "class API:",+ indent 1 "\"\"\"A type-safe client for your Convex API.\"\"\"",+ indent 1 "def __init__(self, client: ConvexClient):",+ indent 2 "self._client = client"+ ]+ body = map (indent 2) initLines ++ definitionLines+ in (unlines (header ++ body), nub nestedModels)++-- Helper to generate Python function arguments and the payload dictionary mapping.+generateArgSignature :: String -> [(String, Schema.ConvexType)] -> (String, String, [String])+generateArgSignature funcName args =+ let results = map (\(n, t) -> (n, toPythonTypeParts (capitalize funcName ++ capitalize n) t)) args+ sigParts = map (\(n, (t, _, _, _)) -> toSnakeCase n ++ ": " ++ t) results+ payloadParts = map (\(n, _) -> "\"" ++ n ++ "\": " ++ toSnakeCase n) results+ nestedModels = concatMap (\(_, (_, _, _, n)) -> n) results+ in (intercalate ", " sigParts, intercalate ", " payloadParts, nestedModels)++-- Helper to get the return type information.+getReturnType :: String -> Schema.ConvexType -> (String, Bool, [String])+getReturnType funcName rt =+ let (pyType, _, _, nested) = toPythonTypeParts (capitalize funcName ++ "Return") rt+ isModel = case rt of+ Schema.VObject _ -> True+ Schema.VArray (Schema.VObject _) -> True+ Schema.VReference _ -> True+ _ -> False+ in (pyType, isModel, nested)++-- Helper to generate a single field line for a Pydantic model.+generateField :: String -> Schema.Field -> (String, [String])+generateField parentClassName field =+ let originalFieldName = Schema.fieldName field+ isSystemField = "_" `isPrefixOf` originalFieldName+ fieldNameSnake = if isSystemField then toSnakeCase (tail originalFieldName) else toSnakeCase originalFieldName+ (pyType, isOpt, isArr, nested) = toPythonTypeParts (parentClassName ++ capitalize originalFieldName) (Schema.fieldType field)++ fieldArgs =+ let defaultArg =+ if isOpt+ then if isArr then "default_factory=list" else "default=None"+ else "..."+ aliasArg = if isSystemField then Just ("alias=\"" ++ originalFieldName ++ "\"") else Nothing+ in intercalate ", " (catMaybes [Just defaultArg, aliasArg])++ fieldDef = fieldNameSnake ++ ": " ++ pyType ++ " = Field(" ++ fieldArgs ++ ")"+ in (indent 1 fieldDef, nested)++-- Core recursive function to generate Python types from the AST.+toPythonTypeParts :: String -> Schema.ConvexType -> (String, Bool, Bool, [String])+toPythonTypeParts nameHint typ = case typ of+ Schema.VString -> ("str", False, False, [])+ Schema.VNumber -> ("float", False, False, [])+ Schema.VInt64 -> ("int", False, False, [])+ Schema.VFloat64 -> ("float", False, False, [])+ Schema.VBoolean -> ("bool", False, False, [])+ Schema.VBytes -> ("bytes", False, False, [])+ Schema.VAny -> ("Any", False, False, [])+ Schema.VNull -> ("None", True, False, [])+ Schema.VId t -> ("Id['" ++ toClassName t ++ "']", False, False, [])+ Schema.VArray inner ->+ let (innerType, isOpt, isArr, nested) = toPythonTypeParts nameHint inner+ in ("list[" ++ innerType ++ "]", isOpt, isArr, nested)+ Schema.VOptional inner ->+ let (innerType, _, innerIsArray, nested) = toPythonTypeParts nameHint inner+ in (innerType ++ " | None", True, innerIsArray, nested)+ Schema.VUnion types ->+ let results = map (toPythonTypeParts nameHint) types+ pyTypes = nub $ map (\(t, _, _, _) -> t) results+ nested = concatMap (\(_, _, _, n) -> n) results+ in (intercalate " | " pyTypes, False, False, nested)+ Schema.VLiteral s -> ("Literal[\"" ++ s ++ "\"]", False, False, [])+ Schema.VReference n -> (n, False, False, [])+ Schema.VObject fields ->+ let className = capitalize nameHint ++ "Object"+ (fieldLines, nested) = unzip $ map (generateField className) (map (\(n, t) -> Schema.Field n t) fields)+ newModel =+ unlines $+ [ "class " ++ className ++ "(BaseModel):",+ unlines fieldLines,+ "",+ indent 1 "class Config:",+ indent 2 "populate_by_name: bool = True"+ ]+ in (className, False, False, concat nested ++ [newModel])+ Schema.VVoid -> ("None", True, False, [])++capitalize :: String -> String+capitalize "" = ""+capitalize (c : cs) = toUpper c : cs++toSingular :: String -> String+toSingular s+ | last s == 's' = capitalize (init s)+ | otherwise = capitalize s++toClassName :: String -> String+toClassName s = capitalize s ++ "Doc"++toSnakeCase :: String -> String+toSnakeCase "" = ""+toSnakeCase (c : cs) = toLower c : go cs+ where+ go (c' : cs')+ | isUpper c' = '_' : toLower c' : go cs'+ | otherwise = c' : go cs'+ go "" = ""
+ src/Backend/Python/Validator.hs view
@@ -0,0 +1,139 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE InstanceSigs #-}+{-# LANGUAGE LambdaCase #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeSynonymInstances #-}++module Backend.Python.Validator (run, PythonValidator (..), PythonValidatorEnv (..)) where++import Control.Monad.Reader+import Convex.Validator+import System.Directory (createDirectoryIfMissing, doesFileExist)+import System.Exit (ExitCode (..))+import System.FilePath ((</>))+import System.IO (hGetContents)+import System.Process (CreateProcess (..), StdStream (..), createProcess, proc, waitForProcess)++newtype PythonValidator a = PythonValidator+ { runPythonValidator :: ReaderT PythonValidatorEnv IO a+ }+ deriving (Functor, Applicative, Monad, MonadIO, MonadReader PythonValidatorEnv, MonadFail)++data PythonValidatorEnv = PythonValidatorEnv {projectPath :: FilePath}++run :: PythonValidatorEnv -> PythonValidator a -> IO a+run renv action = runReaderT (runPythonValidator action) renv++instance Validator PythonValidator where+ -- \| Sets up the Python validation sandbox project.+ setup = do+ config <- ask+ let pythonProjectPath = projectPath config+ let stubsPath = pythonProjectPath </> "stubs"++ -- Create the directory structure.+ liftIO $ createDirectoryIfMissing True stubsPath++ -- Write the mypy.ini config file.+ liftIO $+ doesFileExist (pythonProjectPath </> "mypy.ini") >>= \case+ True -> return () -- If it exists, we don't overwrite it.+ False -> writeFile (pythonProjectPath </> "mypy.ini") mypyIniContent++ -- Write the pydantic stub file.+ liftIO $+ doesFileExist (stubsPath </> "pydantic.pyi") >>= \case+ True -> return () -- If it exists, we don't overwrite it.+ False -> writeFile (stubsPath </> "pydantic.pyi") pydanticStubContent++ -- Write the convex stub file.+ liftIO $+ doesFileExist (stubsPath </> "convex.pyi") >>= \case+ True -> return () -- If it exists, we don't overwrite it.+ False -> writeFile (stubsPath </> "convex.pyi") convexStubContent++ -- Write an empty stub for pydantic_core to satisfy mypy+ liftIO $+ doesFileExist (stubsPath </> "pydantic_core.pyi") >>= \case+ True -> return () -- If it exists, we don't overwrite it.+ False -> writeFile (stubsPath </> "pydantic_core.pyi") ""++ return ()++ -- \| Validates the generated Python code using mypy.+ validate generatedCode = do+ pythonProjectPath <- asks projectPath+ let generatedFilePath = pythonProjectPath </> "generated_api.py"++ -- Write the generated code to the sandbox project.+ liftIO $ writeFile generatedFilePath generatedCode++ -- Run `mypy` on the generated file.+ liftIO $ putStrLn "[Validator] Running 'mypy'..."+ -- We run `mypy .` to make it pick up the mypy.ini config automatically.+ let mypyCmd = (proc "mypy" ["."]) {cwd = Just pythonProjectPath, std_out = CreatePipe, std_err = CreatePipe}+ (_, Just hOut, Just hErr, handle) <- liftIO $ createProcess mypyCmd+ exitCode <- liftIO $ waitForProcess handle++ if exitCode == ExitSuccess+ then do+ liftIO $ putStrLn "[Validator] Validation successful."+ content <- liftIO $ readFile generatedFilePath+ return $ Just content+ else do+ liftIO $ putStrLn "[Validator] Error: 'mypy' failed. The generated code has type errors."+ -- Print both stdout and stderr for comprehensive error reporting.+ liftIO $ hGetContents hOut >>= putStrLn+ liftIO $ hGetContents hErr >>= putStrLn+ return Nothing++-- | Content for the mypy.ini configuration file.+mypyIniContent :: String+mypyIniContent =+ unlines+ [ "[mypy]",+ "python_version = 3.9",+ "disallow_untyped_defs = True",+ "check_untyped_defs = True",+ "warn_return_any = True",+ "ignore_missing_imports = True"+ ]++-- | Minimal stub content for the `pydantic` library.+pydanticStubContent :: String+pydanticStubContent =+ unlines+ [ "from typing import Any, Type",+ "",+ "class BaseModel:",+ " def model_validate(cls: Type, obj: Any) -> Any: ...",+ " def model_dump_json(self, *, indent: int | None = ...) -> str: ...",+ "",+ "def Field(default: Any = ..., *, alias: str | None = ...) -> Any: ...",+ "",+ "class TypeAdapter:",+ " def __init__(self, type: Any) -> None: ...",+ " def validate_python(self, data: Any) -> Any: ...",+ "",+ "class ValidationError(Exception): ..."+ ]++-- | Minimal stub content for the `convex` library.+convexStubContent :: String+convexStubContent =+ unlines+ [ "from typing import Any, Iterator",+ "",+ "class ConvexClient:",+ " def __init__(self, url: str) -> None: ...",+ " def set_admin_auth(self, key: str) -> None: ...",+ " def set_auth(self, token: str) -> None: ...",+ " def query(self, path: str, args: dict[str, Any]) -> Any: ...",+ " def mutation(self, path: str, args: dict[str, Any]) -> Any: ...",+ " def action(self, path: str, args: dict[str, Any]) -> Any: ...",+ " def subscribe(self, path: str, args: dict[str, Any]) -> Iterator[Any]: ...",+ "",+ "class ConvexError(Exception): ..."+ ]
+ src/Backend/Rust.hs view
@@ -0,0 +1,1060 @@+{-# LANGUAGE LambdaCase #-}+{-# LANGUAGE OverloadedStrings #-}++module Backend.Rust+ ( generateRustCode,+ )+where++import qualified Convex.Action.Parser as Action+import qualified Convex.Parser as P+import qualified Convex.Schema.Parser as Schema+import Data.Char (isUpper, toLower, toUpper)+import Data.List (intercalate, isInfixOf, isPrefixOf, nub, stripPrefix)+import qualified Data.Map as Map+import PathTree++-- Helper function to prepend a given number of spaces (4 per level).+indent :: Int -> String -> String+indent n s = replicate (n * 4) ' ' ++ s++-- | Top-level function to generate the complete Rust module source.+generateRustCode :: P.ParsedProject -> String+generateRustCode project =+ unlines+ [ "#![allow(dead_code)]",+ "#![allow(non_snake_case)]",+ "// Generated by the Palaba code generator. DO NOT EDIT.",+ "// Save this file as, for example, `src/convex_api.rs`",+ "// and then add `pub mod convex_api;` to your `src/lib.rs` or `src/main.rs`.",+ "//",+ "// Make sure your `Cargo.toml` contains the following dependencies:",+ "// convex = \"0.1.3\"",+ "// serde = { version = \"1.0\", features = [\"derive\"] }",+ "// serde_json = \"1.0\"",+ "// thiserror = \"1.0\"",+ "// anyhow = \"1.0\"",+ "// futures-util = \"0.3\"",+ "",+ generateRustModuleContent project+ ]++-- | Generates the entire content for a single Rust module file.+generateRustModuleContent :: P.ParsedProject -> String+generateRustModuleContent project =+ let (apiClassCode, nestedFromFuncs) = generateApiClass (P.ppFunctions project)+ in unlines+ [ "use convex::{ConvexClient, FunctionResult, Value};",+ "use futures_util::stream::Stream;",+ "use serde::{Deserialize, Deserializer, Serialize, Serializer};",+ "use serde_json;",+ "use std::collections::BTreeMap;",+ "use std::fmt::{self, Display};",+ "use std::marker::PhantomData;",+ "use std::pin::Pin;",+ "use std::task::{Context, Poll};",+ "",+ stripNewlines generateErrorEnum,+ "",+ stripNewlines generateIdStruct,+ "",+ stripNewlines generateFromConvexValueBoilerplate,+ "",+ stripNewlines generateSubscriptionBoilerplate,+ "",+ stripNewlines apiClassCode, -- API class and all submodules+ "",+ stripNewlines $ generateTypesModule project (nub nestedFromFuncs)+ ]++-- | Generates a Rust error enum using `thiserror`.+generateErrorEnum :: String+generateErrorEnum =+ unlines+ [ "/// Represents all possible errors that can occur when interacting with the API.",+ "#[derive(thiserror::Error, Debug)]",+ "pub enum ApiError {",+ indent 1 "#[error(\"Convex client error: {0}\")]",+ indent 1 "ConvexClientError(String),",+ "",+ indent 1 "#[error(\"Convex function error: {0}\")]",+ indent 1 "ConvexFunctionError(String),",+ "",+ indent 1 "#[error(\"Failed to deserialize response: {0}\")]",+ indent 1 "DeserializationError(#[from] serde_json::Error),",+ "",+ indent 1 "#[error(\"Unexpected null value returned from a non-nullable function\")]",+ indent 1 "UnexpectedNullError,",+ "}",+ "",+ "impl From<anyhow::Error> for ApiError {",+ indent 1 "fn from(err: anyhow::Error) -> Self {",+ indent 2 "ApiError::ConvexClientError(err.to_string())",+ indent 1 "}",+ "}"+ ]++-- | Generates the strongly-typed `Id<T>` struct.+generateIdStruct :: String+generateIdStruct =+ unlines+ [ "/// A strongly-typed Convex document ID.",+ "#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]",+ "pub struct Id<T> {",+ indent 1 "id: String,",+ indent 1 "_phantom: PhantomData<T>,",+ "}",+ "",+ "impl<T> Default for Id<T> {",+ indent 1 "fn default() -> Self {",+ indent 2 "Self { id: String::new(), _phantom: PhantomData }",+ indent 1 "}",+ "}",+ "",+ "impl<T> Id<T> {",+ indent 1 "pub fn new(id: String) -> Self {",+ indent 2 "Self { id, _phantom: PhantomData }",+ indent 1 "}",+ "}",+ "",+ "impl<T> Display for Id<T> {",+ indent 1 "fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {",+ indent 2 "write!(f, \"{}\", self.id)",+ indent 1 "}",+ "}",+ "",+ "impl<T> Serialize for Id<T> {",+ indent 1 "fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>",+ indent 1 "where",+ indent 2 "S: Serializer,",+ indent 1 "{",+ indent 2 "serializer.serialize_str(&self.id)",+ indent 1 "}",+ "}",+ "",+ "impl<'de, T> Deserialize<'de> for Id<T> {",+ indent 1 "fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>",+ indent 1 "where",+ indent 2 "D: Deserializer<'de>,",+ indent 1 "{",+ indent 2 "let id = String::deserialize(deserializer)?;",+ indent 2 "Ok(Id::new(id))",+ indent 1 "}",+ "}",+ "",+ "impl<T> From<Id<T>> for Value {",+ indent 1 "fn from(val: Id<T>) -> Self {",+ indent 2 "Value::String(val.id)",+ indent 1 "}",+ "}",+ "impl<T> TryFrom<Value> for Id<T> {",+ indent 1 "type Error = ApiError;",+ "",+ indent 1 "fn try_from(value: Value) -> Result<Self, Self::Error> {",+ indent 2 "if let Value::String(id) = value {",+ indent 3 "Ok(Id::new(id))",+ indent 2 "} else {",+ indent 3 "Err(ApiError::ConvexClientError(",+ indent 4 "\"Expected a string for Id\".to_string(),",+ indent 3 "))",+ indent 2 "}",+ indent 1 "}",+ "}",+ "",+ indent 0 "impl<T> FromConvexValue for Id<T> {",+ indent 1 "fn from_convex(value: Value) -> Result<Self, ApiError> {",+ indent 2 "Id::try_from(value).map_err(ApiError::from)",+ indent 1 "}",+ indent 0 "}"+ ]++-- | Generates the generic TypedSubscription struct and its Stream implementation.+generateSubscriptionBoilerplate :: String+generateSubscriptionBoilerplate =+ unlines+ [ "/// A type-safe, auto-deserializing stream of updates from a Convex query subscription.",+ "#[derive(Debug)]",+ "pub struct TypedSubscription<T> {",+ indent 1 "raw_subscription: convex::QuerySubscription,",+ indent 1 "_phantom: PhantomData<T>,",+ "}",+ "",+ "impl<T> Stream for TypedSubscription<T>",+ "where",+ indent 1 "T: FromConvexValue,",+ "{",+ indent 1 "type Item = Result<T, ApiError>;",+ "",+ indent 1 "fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {",+ indent 2 "let raw_sub_pin: Pin<&mut _> = unsafe { self.map_unchecked_mut(|s| &mut s.raw_subscription) };",+ indent 2 "match raw_sub_pin.poll_next(cx) {",+ indent 3 "Poll::Ready(Some(result)) => {",+ indent 4 "let item = match result {",+ indent 5 "FunctionResult::Value(value) => T::from_convex(value),",+ indent 5 "FunctionResult::ErrorMessage(s) => Err(ApiError::ConvexFunctionError(s)),",+ indent 5 "FunctionResult::ConvexError(err) => Err(ApiError::ConvexClientError(err.to_string())),",+ indent 4 "};",+ indent 4 "Poll::Ready(Some(item))",+ indent 3 "}",+ indent 3 "Poll::Ready(None) => Poll::Ready(None),",+ indent 3 "Poll::Pending => Poll::Pending,",+ indent 2 "}",+ indent 1 "}",+ "}"+ ]++generateFromConvexValueBoilerplate :: String+generateFromConvexValueBoilerplate =+ unlines+ [ "pub trait FromConvexValue: Sized {",+ " fn from_convex(value: Value) -> Result<Self, ApiError>;",+ "}",+ "",+ "impl FromConvexValue for bool {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Boolean(b) => Ok(b),",+ " _ => Err(ApiError::ConvexClientError(\"Expected bool\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for i64 {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Int64(i) => Ok(i),",+ " _ => Err(ApiError::ConvexClientError(\"Expected i64\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for i32 {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Int64(i) => i",+ " .try_into()",+ " .map_err(|_| ApiError::ConvexClientError(\"i64 out of range for i32\".into())),",+ " _ => Err(ApiError::ConvexClientError(\"Expected i64\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for f64 {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Float64(f) => Ok(f),",+ " _ => Err(ApiError::ConvexClientError(\"Expected f64\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for f32 {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Float64(f) => Ok(f as f32),",+ " _ => Err(ApiError::ConvexClientError(\"Expected f64\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl<T> FromConvexValue for Vec<T>",+ "where",+ " T: FromConvexValue,",+ "{",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Array(arr) => arr",+ " .into_iter()",+ " .map(T::from_convex)",+ " .collect::<Result<Vec<T>, ApiError>>(),",+ " _ => Err(ApiError::ConvexClientError(\"Expected array\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl<T> FromConvexValue for Option<T>",+ "where",+ " T: FromConvexValue,",+ "{",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Null => Ok(None),",+ " other => T::from_convex(other).map(Some),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for Vec<u8> {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::Bytes(bytes) => Ok(bytes),",+ " _ => Err(ApiError::ConvexClientError(\"Expected bytes\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for String {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " match value {",+ " Value::String(s) => Ok(s),",+ " _ => Err(ApiError::ConvexClientError(\"Expected string\".into())),",+ " }",+ " }",+ "}",+ "",+ "impl FromConvexValue for serde_json::Value {",+ " fn from_convex(value: Value) -> Result<Self, ApiError> {",+ " Ok(value.into())",+ " }",+ "}",+ ""+ ]++generateApiClass :: [Action.ConvexFunction] -> (String, [String])+generateApiClass funcs =+ let tree = buildPathTree funcs+ (structDefs, implDef, nested) = generateApiStructure "Api" tree+ in ( unlines+ [ "pub struct Api {",+ indent 1 "pub client: ConvexClient,",+ "}",+ structDefs,+ implDef+ ],+ nested+ )++generateApiStructure :: String -> PathTree -> (String, String, [String])+generateApiStructure parentName (DirNode dirMap) =+ let (structs, impls, nested) = unzip3 $ map (uncurry processEntry) (Map.toList dirMap)+ (accessors, functions) = partitionEntries (Map.toList dirMap)+ in ( unlines structs,+ unlines+ [ "impl" ++ (if parentName == "Api" then "" else "<'a>") ++ " " ++ parentName ++ (if parentName == "Api" then "" else "<'a>") ++ " {",+ if parentName == "Api" then indent 1 "pub fn new(client: ConvexClient) -> Self {\n Self { client }\n }" else "",+ unlines (map generateAccessorMethod accessors),+ unlines (concatMap generateMethodsForEntry functions),+ "}"+ ]+ ++ unlines impls,+ concat nested+ )+ where+ processEntry name (DirNode subDir) =+ let structName = toPascalCase name+ (subStructs, subImpls, nestedFromSub) = generateApiStructure structName (DirNode subDir)+ structDef =+ unlines+ [ "pub struct " ++ structName ++ "<'a> {",+ indent 1 "client: &'a mut ConvexClient,",+ "}"+ ]+ in (unlines [structDef, subStructs], subImpls, nestedFromSub)+ processEntry _ (FuncNode func) =+ let (_, nestedFromFunc) = generateFunction func+ (_, nestedFromSub) = if Action.funcType func == Action.Query then generateSubscriptionFunction func else ("", [])+ in ("", "", nestedFromFunc ++ nestedFromSub)++ partitionEntries =+ foldl+ ( \(ds, fs) (name, node) -> case node of+ DirNode _ -> ((name, node) : ds, fs)+ FuncNode _ -> (ds, (name, node) : fs)+ )+ ([], [])++ generateAccessorMethod (name, _) =+ let structName = toPascalCase name+ methodName = toSnakeCase name+ in unlines+ [ indent 1 ("pub fn " ++ methodName ++ "(&mut self) -> " ++ structName ++ "<'_> {"),+ indent 2 (structName ++ " { client: &mut self.client }"),+ indent 1 "}"+ ]++ generateMethodsForEntry (_, FuncNode func) =+ let (queryDef, _) = generateFunction func+ (subDef, _) =+ if Action.funcType func == Action.Query+ then generateSubscriptionFunction func+ else ("", [])+ in [queryDef, subDef]+ generateMethodsForEntry _ = []+generateApiStructure _ _ = ("", "", [])++generateFunction :: Action.ConvexFunction -> (String, [String])+generateFunction func =+ let funcName = Action.funcName func+ (argSignature, nestedFromArgs) = generateArgSignature funcName (Action.funcArgs func)+ funcNameSnake = toSnakeCase funcName+ (returnHint, isNullable, nestedFromReturn) = getReturnType funcName (Action.funcReturn func)+ handlerCall = case Action.funcType func of+ Action.Query -> "query"+ Action.Mutation -> "mutation"+ Action.Action -> "action"+ fullFuncPath = Action.funcPath func ++ ":" ++ funcName+ btreemapConstruction = generateBTreeMap (Action.funcArgs func)+ returnHandling = generateReturnHandling returnHint isNullable+ funcCode =+ unlines+ [ indent 1 ("/// Wraps the `" ++ fullFuncPath ++ "` " ++ show (Action.funcType func) ++ "."),+ indent 1 ("pub async fn " ++ funcNameSnake ++ "(&mut self, " ++ argSignature ++ ") -> Result<" ++ returnHint ++ ", ApiError> {"),+ btreemapConstruction,+ indent 2 ("let result = self.client." ++ handlerCall ++ "(\"" ++ fullFuncPath ++ "\", btmap).await?;"),+ returnHandling,+ indent 1 "}"+ ]+ in (funcCode, nestedFromArgs ++ nestedFromReturn)++generateSubscriptionFunction :: Action.ConvexFunction -> (String, [String])+generateSubscriptionFunction func =+ let funcName = Action.funcName func+ (argSignature, nestedFromArgs) = generateArgSignature funcName (Action.funcArgs func)+ funcNameSnake = "subscribe_" ++ toSnakeCase funcName+ (returnHint, _, nestedFromReturn) = getReturnType funcName (Action.funcReturn func)+ fullFuncPath = Action.funcPath func ++ ":" ++ funcName+ btreemapConstruction = generateBTreeMap (Action.funcArgs func)+ funcCode =+ unlines+ [ indent 1 ("/// Subscribes to the `" ++ fullFuncPath ++ "` query."),+ indent 1 ("pub async fn " ++ funcNameSnake ++ "(&mut self, " ++ argSignature ++ ") -> Result<TypedSubscription<" ++ returnHint ++ ">, ApiError> {"),+ btreemapConstruction,+ indent 2 ("let raw_subscription = self.client.subscribe(\"" ++ fullFuncPath ++ "\", btmap).await?;"),+ indent 2 "Ok(TypedSubscription {",+ indent 3 "raw_subscription,",+ indent 3 "_phantom: PhantomData,",+ indent 2 "})",+ indent 1 "}"+ ]+ in (funcCode, nestedFromArgs ++ nestedFromReturn)++generateTypesModule :: P.ParsedProject -> [String] -> String+generateTypesModule project nestedFromFuncs =+ let (tableCode, nestedFromTables) = generateAllTables (P.ppSchema project)+ (constantsCode, nestedFromConstants) = generateAllConstants (P.ppConstants project)+ allNestedCode = nub (nestedFromTables ++ nestedFromConstants ++ nestedFromFuncs)+ in unlines+ [ "pub mod types {",+ indent 1 "use super::*;",+ "",+ tableCode,+ constantsCode,+ unlines allNestedCode,+ "}"+ ]++generateAllTables :: Schema.Schema -> (String, [String])+generateAllTables (Schema.Schema tables) =+ let (tableCodes, nested) = unzip $ map generateTableStruct tables+ in (unlines tableCodes, concat nested)++generateTableStruct :: Schema.Table -> (String, [String])+generateTableStruct table =+ let className = toPascalCase (Schema.tableName table) ++ "Doc"+ (fieldLines, nestedFromFields) = unzip $ map (generateField className) (Schema.tableFields table)+ in ( unlines+ [ "#[derive(Default, Serialize, Deserialize, Debug, Clone, PartialEq)]",+ ("pub struct " ++ className ++ " {"),+ indent 1 "#[serde(default)]",+ indent 1 ("pub _id: Id<" ++ className ++ ">,"),+ indent 1 "#[serde(default)]",+ indent 1 "#[serde(rename = \"_creationTime\")]",+ indent 1 "pub creation_time: f64,",+ unlines fieldLines,+ "}"+ ],+ concat nestedFromFields+ )++generateAllConstants :: Map.Map String Schema.ConvexType -> (String, [String])+generateAllConstants constants =+ let (constCodes, nested) = unzip $ map (uncurry generateConstant) (Map.toList constants)+ in (unlines constCodes, concat nested)++generateConstant :: String -> Schema.ConvexType -> (String, [String])+generateConstant name u@(Schema.VUnion literals)+ | all Schema.isLiteral literals =+ let enumName = toPascalCase name+ enumFromConvexValueImpl = generateFromConvexValueImplEnum ("types::" ++ enumName) literals+ variantNames = map Schema.getLiteralString literals+ buildVariantLines [] = []+ buildVariantLines (first : rest) =+ (indent 2 "#[default]\n" ++ indent 2 ("#[serde(rename = \"" ++ first ++ "\")]\n") ++ indent 2 (toPascalCase first ++ ","))+ : map (\v -> indent 2 ("#[serde(rename = \"" ++ v ++ "\")]\n") ++ indent 2 (toPascalCase v ++ ",")) rest+ code =+ unlines+ [ indent 1 "#[derive(Default, Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]",+ indent 1 ("pub enum " ++ enumName ++ " {"),+ unlines $ buildVariantLines variantNames,+ indent 1 "}",+ "",+ enumFromConvexValueImpl,+ ""+ ]+ in (code, [])+ | otherwise =+ let (rustTypeName, nested) = toRustType name u+ in (indent 1 ("pub type " ++ toPascalCase name ++ " = " ++ rustTypeName ++ ";"), nested)+generateConstant name t =+ let (rustTypeName, nested) = toRustType name t+ in (indent 1 ("pub type " ++ toPascalCase name ++ " = " ++ rustTypeName ++ ";"), nested)++generateField :: String -> Schema.Field -> (String, [String])+generateField nameHint field =+ let fieldNameSnake = toSnakeCase (Schema.fieldName field)+ (rustType, nested) = toRustType (nameHint ++ capitalize (Schema.fieldName field)) (Schema.fieldType field)+ serdeRename = if fieldNameSnake /= Schema.fieldName field then indent 2 ("#[serde(rename = \"" ++ Schema.fieldName field ++ "\")]\n") else ""+ serdeAttrs =+ let defaultAttr = "default"+ skipAttr =+ if needsOptionalWrapper (Schema.fieldType field)+ then Just "skip_serializing_if = \"Option::is_none\""+ else Nothing+ allAttrs = case skipAttr of+ Just s -> [defaultAttr, s]+ Nothing -> [defaultAttr]+ in indent 2 ("#[serde(" ++ intercalate ", " allAttrs ++ ")]")+ fieldLine = serdeRename ++ serdeAttrs ++ "\n" ++ indent 2 ("pub " ++ fieldNameSnake ++ ": " ++ rustType ++ ",")+ in (fieldLine, nested)++generateArgSignature :: String -> [(String, Schema.ConvexType)] -> (String, [String])+generateArgSignature funcName args =+ let results = map (\(n, t) -> toRustType (funcName ++ capitalize n) t) args+ sigParts = zipWith (\(n, _) (ty, _) -> toSnakeCase n ++ ": " ++ toRustBorrowType ty) args results+ nestedModels = concatMap snd results+ in (intercalate ", " sigParts, nestedModels)++generateBTreeMap :: [(String, Schema.ConvexType)] -> String+generateBTreeMap [] = indent 2 "let btmap = BTreeMap::new();"+generateBTreeMap btmap =+ let buildStmts (name, convexType) =+ let varName = toSnakeCase name+ in case convexType of+ Schema.VObject _ -> indent 1 ("btmap.insert(\"" ++ name ++ "\".to_string(), " ++ varName ++ ".to_convex_value()?);")+ Schema.VOptional innerConvexType -> indent 1 ("if let Some(v) = " ++ varName ++ " { btmap.insert(\"" ++ name ++ "\".to_string(), " ++ fieldToConvexValue ("v", innerConvexType) ++ "); }")+ _ -> indent 1 ("btmap.insert(\"" ++ name ++ "\".to_string(), " ++ fieldToConvexValue (varName, convexType) ++ ");")+ in unlines+ [ indent 2 "let mut btmap = BTreeMap::new();",+ unlines $ map buildStmts btmap+ ]++fieldToConvexValue :: (String, Schema.ConvexType) -> String+fieldToConvexValue (fieldName, t) =+ let fieldNameSnake = toSnakeCase fieldName+ valueExpr = innerValueToConvexNonOptional fieldNameSnake t+ in valueExpr++toClonedValue :: String -> Schema.ConvexType -> String+toClonedValue varName (Schema.VString) = varName ++ ".to_string()"+toClonedValue varName t+ | isPassedByCopy t = "*" ++ varName+ | otherwise = varName ++ ".clone()"++isPassedByCopy :: Schema.ConvexType -> Bool+isPassedByCopy Schema.VNumber = True+isPassedByCopy Schema.VInt64 = True+isPassedByCopy Schema.VFloat64 = True+isPassedByCopy Schema.VBoolean = True+isPassedByCopy _ = False++getReturnType :: String -> Schema.ConvexType -> (String, Bool, [String])+getReturnType funcName rt =+ let (baseType, nested) = toRustType (funcName ++ "Return") rt+ isNullable = needsOptionalWrapper rt+ in if baseType == "()"+ then ("()", False, nested)+ else (baseType, isNullable, nested)++generateReturnHandling :: String -> Bool -> String+generateReturnHandling "()" _ =+ unlines+ [ indent 2 "match result {",+ indent 3 "FunctionResult::Value(_) => Ok(()),",+ indent 3 "FunctionResult::ErrorMessage(s) => Err(ApiError::ConvexFunctionError(s)),",+ indent 3 "FunctionResult::ConvexError(err) => Err(ApiError::ConvexClientError(err.to_string())),",+ indent 2 "}"+ ]+generateReturnHandling _ isNullable =+ if isNullable+ then+ unlines+ [ indent 2 "match result {",+ indent 3 "FunctionResult::Value(val) => Ok(FromConvexValue::from_convex(val.clone())?),",+ indent 3 "FunctionResult::ErrorMessage(s) => Err(ApiError::ConvexFunctionError(s)),",+ indent 3 "FunctionResult::ConvexError(err) => Err(ApiError::ConvexClientError(err.to_string())),",+ indent 2 "}"+ ]+ else+ unlines+ [ indent 2 "match result {",+ indent 3 "FunctionResult::Value(Value::Null) => Err(ApiError::UnexpectedNullError),",+ indent 3 "FunctionResult::Value(val) => Ok(FromConvexValue::from_convex(val.clone())?),",+ indent 3 "FunctionResult::ErrorMessage(s) => Err(ApiError::ConvexFunctionError(s)),",+ indent 3 "FunctionResult::ConvexError(err) => Err(ApiError::ConvexClientError(err.to_string())),",+ indent 2 "}"+ ]++generateToConvexValueImpl :: String -> [(String, Schema.ConvexType)] -> String+generateToConvexValueImpl structName fields =+ let buildMapInserts (fieldName, fieldType) =+ let conversionBlock = generateFieldToConvexValue (fieldName, fieldType)+ in indent 3 conversionBlock+ mapInserts = unlines $ map buildMapInserts fields+ in unlines+ [ "impl " ++ structName ++ " {",+ indent 1 "pub fn to_convex_value(&self) -> Result<Value, ApiError> {",+ indent 2 "let mut btmap = BTreeMap::new();",+ mapInserts,+ indent 2 "Ok(Value::Object(btmap))",+ indent 1 "}",+ "}"+ ]++generateFromConvexValueImplEnum :: String -> [Schema.ConvexType] -> String+generateFromConvexValueImplEnum structName fields =+ unlines+ [ "impl TryFrom<Value> for " ++ structName ++ " {",+ indent 1 "type Error = ApiError;",+ indent 1 "fn try_from(value: Value) -> Result<Self, Self::Error> {",+ indent 2 "if let Value::String(s) = &value {",+ indent 3 "return match s.as_str() {",+ unlines . map (indent 4) $ generateEnumMatchCases structName fields,+ indent 2 "}",+ indent 1 "}",+ indent 2 "Err(ApiError::ConvexClientError(\"Expected a string for " ++ structName ++ "\".to_string()))",+ indent 1 "}",+ indent 1 "}",+ "",+ indent 1 $ "impl FromConvexValue for " ++ structName ++ " {",+ indent 2 $ "fn from_convex(value: Value) -> Result<Self, ApiError> {",+ indent 3 $ structName ++ "::try_from(value)",+ indent 2 $ "}",+ indent 1 $ "}",+ ""+ ]++generateEnumMatchCases :: String -> [Schema.ConvexType] -> [String]+generateEnumMatchCases structName fields =+ let cases =+ map+ ( \case+ (Schema.VLiteral s) -> "\"" ++ s ++ "\" => Ok(" ++ structName ++ "::" ++ toPascalCase s ++ "),"+ _ -> error "Expected a literal for enum field"+ )+ fields+ in cases ++ ["x => Err(ApiError::ConvexClientError(\"Expected one of " ++ enumValues ++ " for " ++ structName ++ " but got \".to_string() + &x.to_string()))"]+ where+ enumValues = "[" ++ (intercalate ", " $ map Schema.getLiteralString fields) ++ "]"++-- #[derive(Default, Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]+-- pub enum AccessLevelEnum {+-- #[default]+-- #[serde(rename = "read")]+-- Read,+-- #[serde(rename = "edit")]+-- Edit,+-- }+--+-- impl From<Value> for AccessLevelEnum {+-- fn from(value: Value) -> Self {+-- match value {+-- Value::String(s) => match s.as_str() {+-- "read" => AccessLevelEnum::Read,+-- "edit" => AccessLevelEnum::Edit,+-- _ => panic!("Unknown AccessLevelEnum value"),+-- },+-- _ => panic!("Expected a string for AccessLevelEnum"),+-- }+-- }+-- }++generateFromConvexValueImpl :: String -> [(String, Schema.ConvexType)] -> String+generateFromConvexValueImpl structName fields =+ unlines+ [ "impl TryFrom<Value> for " ++ structName ++ " {",+ indent 1 "type Error = ApiError;",+ indent 1 "fn try_from(value: Value) -> Result<Self, Self::Error> {",+ indent 2 "let obj = match value {",+ indent 3 "Value::Object(map) => map,",+ indent 3 "_ => return Err(ApiError::ConvexClientError(\"Expected object\".to_string())),",+ indent 2 "};",+ unlines $ map (indent 2 . generateAccessorFnFromConvexValue structName) fields,+ unlines $ map (indent 2) $ generateFromConvexValueResult structName fields,+ indent 1 "}",+ "}",+ "",+ indent 1 $ "impl FromConvexValue for " ++ structName ++ " {",+ indent 2 $ "fn from_convex(value: Value) -> Result<Self, ApiError> {",+ indent 3 $ structName ++ "::try_from(value)",+ indent 2 $ "}",+ indent 1 $ "}",+ ""+ ]++generateFromConvexValueResult :: String -> [(String, Schema.ConvexType)] -> [String]+generateFromConvexValueResult structName fields =+ let fieldNames = map (toSnakeCase . fst) fields+ fieldAccessors = map (\name -> "get_" ++ name) fieldNames+ setterStatements =+ zipWith+ ( \name getter -> case name of+ "_creation_time" -> indent 3 $ "_creation_time: " ++ getter ++ "(&obj, \"_creationTime\")?"+ n -> indent 3 $ n ++ ": " ++ getter ++ "(&obj, \"" ++ n ++ "\")?"+ )+ fieldNames+ fieldAccessors+ in [ indent 2 ("Ok(" ++ structName ++ " {"),+ indent 3 $ intercalate ",\n" $ setterStatements,+ indent 2 "})"+ ]++-- Ok(types::GetAssetsReturnObject {+-- _id: Id::new(get__id(&obj, "_id")?),+-- _creation_time: get__creation_time(&obj, "_creationTime")?,+-- project_id: Id::new(get_project_id(&obj, "project_id")?),+-- asset_name: get_asset_name(&obj, "asset_name")?,+-- asset_essence_mtime: get_asset_essence_mtime(&obj, "asset_essence_mtime")?,+-- link_metadata: get_link_metadata(&obj, "link_metadata")?,+-- })++generateAccessorFnFromConvexValue :: String -> (String, Schema.ConvexType) -> String+generateAccessorFnFromConvexValue structName (fieldName, Schema.VOptional fieldType) =+ let fieldNameSnake = toSnakeCase fieldName+ getterName = "get_" ++ fieldNameSnake+ (fieldTypeStr, _) = case stripPrefix (reverse "Object") (reverse structName) of+ Just cleanStructName -> toRustType (reverse cleanStructName ++ capitalize fieldName) fieldType+ Nothing -> error "Expected struct name to end with 'Object' for optional field"+ in unlines+ [ indent 0 $ "fn " ++ getterName ++ "(map: &BTreeMap<String, Value>, key: &str) -> Result<Option<" ++ fieldTypeStr ++ ">, ApiError> {",+ indent 1 $ "match map.get(key) {",+ indent 2 $ "Some(v) => {",+ indent 3 $ "Ok(Some(FromConvexValue::from_convex(v.clone())?))",+ indent 2 $ "}",+ indent 2 $ "_ => Ok(None),",+ indent 1 $ "}",+ indent 0 $ "}",+ ""+ ]+generateAccessorFnFromConvexValue structName (fieldName, fieldType) =+ let fieldNameSnake = toSnakeCase fieldName+ getterName = "get_" ++ fieldNameSnake+ (fieldTypeStr, _) = case stripPrefix (reverse "Object") (reverse structName) of+ Just cleanStructName -> toRustType (reverse cleanStructName ++ capitalize fieldName) fieldType+ Nothing -> error "Expected struct name to end with 'Object' for optional field"+ in unlines+ [ indent 0 $ "fn " ++ getterName ++ "(map: &BTreeMap<String, Value>, key: &str) -> Result<" ++ fieldTypeStr ++ ", ApiError> {",+ indent 1 $ "match map.get(key) {",+ indent 2 $ "Some(v) => {",+ indent 3 $ "Ok(FromConvexValue::from_convex(v.clone())?)",+ indent 2 $ "}",+ indent 2 $ "_ => return Err(ApiError::ConvexClientError(format!(\"Expected field (" ++ fieldTypeStr ++ ") '{}' not found\", key))),",+ indent 1 $ "}",+ indent 0 $ "}",+ ""+ ]++-- An example implementation:+--+-- pub struct GetAssetsReturnObject {+-- #[serde(default)]+-- pub _id: Id<types::AssetsDoc>,+-- #[serde(rename = "_creationTime")]+-- #[serde(default)]+-- pub _creation_time: f64,+-- #[serde(default)]+-- pub project_id: Id<types::ProjectsDoc>,+-- #[serde(default)]+-- pub asset_name: String,+-- #[serde(default)]+-- pub link_metadata: types::GetAssetsReturnLinkMetadataObject,+-- #[serde(default)]+-- pub asset_essence_mtime: i64,+-- }+--+-- impl TryFrom<Value> for types::GetAssetsReturnObject {+-- type Error = ApiError;+--+-- fn try_from(value: Value) -> Result<Self, Self::Error> {+-- let obj = match value {+-- Value::Object(map) => map,+-- _ => return Err(ApiError::ConvexClientError("Expected object".to_string())),+-- };+--+-- fn get__id(map: &BTreeMap<String, Value>, key: &str) -> Result<String, ApiError> {+-- match map.get(key) {+-- Some(Value::String(s)) => Ok(s.clone()),+-- _ => Err(ApiError::ConvexClientError(format!(+-- "Expected string for field '{}'",+-- key+-- ))),+-- }+-- }+--+-- fn get_project_id(map: &BTreeMap<String, Value>, key: &str) -> Result<String, ApiError> {+-- match map.get(key) {+-- Some(Value::String(s)) => Ok(s.clone()),+-- _ => Err(ApiError::ConvexClientError(format!(+-- "Expected string for field '{}'",+-- key+-- ))),+-- }+-- }+--+-- fn get__creation_time(map: &BTreeMap<String, Value>, key: &str) -> Result<f64, ApiError> {+-- match map.get(key) {+-- Some(Value::Float64(f)) => Ok(*f),+-- _ => Err(ApiError::ConvexClientError(format!(+-- "Expected float64 for field '{}'",+-- key+-- ))),+-- }+-- }+--+-- fn get_asset_essence_mtime(+-- map: &BTreeMap<String, Value>,+-- key: &str,+-- ) -> Result<i64, ApiError> {+-- match map.get(key) {+-- Some(Value::Int64(i)) => Ok(*i),+-- _ => Err(ApiError::ConvexClientError(format!(+-- "Expected int64 for field '{}'",+-- key+-- ))),+-- }+-- }+--+-- fn get_asset_name(map: &BTreeMap<String, Value>, key: &str) -> Result<String, ApiError> {+-- match map.get(key) {+-- Some(Value::String(s)) => Ok(s.clone()),+-- _ => Err(ApiError::ConvexClientError(format!(+-- "Expected string for field '{}'",+-- key+-- ))),+-- }+-- }+--+-- fn get_link_metadata(+-- map: &BTreeMap<String, Value>,+-- key: &str,+-- ) -> Result<types::GetAssetsReturnLinkMetadataObject, ApiError> {+-- match map.get(key) {+-- Some(Value::Object(inner)) => {+-- let length = match inner.get("length") {+-- Some(Value::Int64(i)) => *i,+-- _ => {+-- return Err(ApiError::ConvexClientError(+-- "Expected int64 for 'length'".into(),+-- ));+-- }+-- };+-- let sample_rate = match inner.get("sample_rate") {+-- Some(Value::Int64(i)) => *i,+-- _ => {+-- return Err(ApiError::ConvexClientError(+-- "Expected int64 for 'sample_rate'".into(),+-- ));+-- }+-- };+-- let summary = match inner.get("summary") {+-- Some(Value::Bytes(b)) => b.clone(),+-- _ => {+-- return Err(ApiError::ConvexClientError(+-- "Expected bytes for 'summary'".into(),+-- ));+-- }+-- };+-- Ok(types::GetAssetsReturnLinkMetadataObject {+-- length,+-- sample_rate,+-- summary,+-- })+-- }+-- _ => Err(ApiError::ConvexClientError(+-- "Expected object for 'link_metadata'".into(),+-- )),+-- }+-- }+--+-- Ok(types::GetAssetsReturnObject {+-- _id: Id::new(get__id(&obj, "_id")?),+-- _creation_time: get__creation_time(&obj, "_creationTime")?,+-- project_id: Id::new(get_project_id(&obj, "project_id")?),+-- asset_name: get_asset_name(&obj, "asset_name")?,+-- asset_essence_mtime: get_asset_essence_mtime(&obj, "asset_essence_mtime")?,+-- link_metadata: get_link_metadata(&obj, "link_metadata")?,+-- })+-- }+-- }++-- Some(Value::Array(v)) => Ok(Some(+-- v.iter()+-- .map(|item| item.clone().try_into())+-- .collect::<Result<Vec<_>, ApiError>>()?,+-- )),++generateFieldToConvexValue :: (String, Schema.ConvexType) -> String+generateFieldToConvexValue (fieldName, Schema.VOptional inner) =+ let fieldNameSnake = toSnakeCase fieldName+ -- `v` is the unwrapped value from the Option.+ valueExpr = innerValueToConvexOptional "v" inner+ in unlines+ [ "if let Some(v) = &self." ++ fieldNameSnake ++ " {",+ indent 1 ("btmap.insert(\"" ++ fieldName ++ "\".to_string(), " ++ valueExpr ++ ");"),+ "}"+ ]+generateFieldToConvexValue (fieldName, fieldType) =+ let fieldNameSnake = toSnakeCase fieldName+ valueExpr = innerValueToConvexNonOptional ("self." ++ fieldNameSnake) fieldType+ in "btmap.insert(\"" ++ fieldName ++ "\".to_string(), " ++ valueExpr ++ ");"++-- | Generates the conversion for a non-optional inner value.+innerValueToConvexOptional :: String -> Schema.ConvexType -> String+innerValueToConvexOptional varName (Schema.VArray inner) =+ let itemConversion = innerValueToConvexArray "item" inner+ -- We can check `isFallible` by looking for `Value::try_from` or `?` in the conversion.+ isFallible = "?" `isInfixOf` itemConversion || "Value::try_from" `isInfixOf` itemConversion || "to_convex_value" `isInfixOf` itemConversion+ in if isFallible+ then "Value::Array(" ++ varName ++ ".iter().map(|item| " ++ itemConversion ++ ").collect::<Result<Vec<_>, _>>()?)"+ else "Value::Array(" ++ varName ++ ".iter().map(|item| " ++ itemConversion ++ ").collect())"+innerValueToConvexOptional varName (Schema.VObject _) =+ varName ++ ".to_convex_value()"+innerValueToConvexOptional varName t+ | isComplexForBTreeMap t = "Value::try_from(serde_json::to_value(" ++ toClonedValue varName t ++ ").unwrap_or(\"unable to serialize\".into()))?"+ | otherwise = "Value::from(" ++ toClonedValue varName t ++ ")"++innerValueToConvexArray :: String -> Schema.ConvexType -> String+innerValueToConvexArray varName (Schema.VArray inner) =+ let itemConversion = innerValueToConvexArray "item" inner+ isFallible = "?" `isInfixOf` itemConversion || "Value::try_from" `isInfixOf` itemConversion || "to_convex_value" `isInfixOf` itemConversion+ in if isFallible+ then "Value::Array(" ++ varName ++ ".iter().map(|item| " ++ itemConversion ++ ").collect::<Result<Vec<_>, _>>()?)"+ else "Value::Array(" ++ varName ++ ".iter().map(|item| " ++ itemConversion ++ ").collect())"+innerValueToConvexArray varName (Schema.VObject _) =+ varName ++ ".to_convex_value()"+innerValueToConvexArray varName t+ | isComplexForBTreeMap t = "Value::try_from(serde_json::to_value(" ++ toClonedValue varName t ++ ").unwrap_or(\"unable to serialize\".into()))"+ | otherwise = "Value::from(" ++ toClonedValue varName t ++ ")"++innerValueToConvexNonOptional :: String -> Schema.ConvexType -> String+innerValueToConvexNonOptional varName (Schema.VArray inner) =+ let itemConversion = innerValueToConvexArray "item" inner+ isFallible = "?" `isInfixOf` itemConversion || "Value::try_from" `isInfixOf` itemConversion || "to_convex_value" `isInfixOf` itemConversion+ in if isFallible+ then "Value::Array(" ++ varName ++ ".iter().map(|item| " ++ itemConversion ++ ").collect::<Result<Vec<_>, _>>()?)"+ else "Value::Array(" ++ varName ++ ".iter().map(|item| " ++ itemConversion ++ ").collect())"+innerValueToConvexNonOptional varName (Schema.VBytes) = "Value::from(" ++ varName ++ ".to_vec())"+innerValueToConvexNonOptional varName (Schema.VObject _) = "Value::from(" ++ varName ++ ".to_convex_value()?)"+innerValueToConvexNonOptional varName t+ | isComplexForBTreeMap t = "Value::try_from(serde_json::to_value(" ++ toClonedValue varName t ++ ").unwrap_or(\"unable to serialize\".into()))?"+ | otherwise = "Value::from(" ++ toClonedValueNonOptional varName t ++ ")"++toClonedValueNonOptional :: String -> Schema.ConvexType -> String+toClonedValueNonOptional varName (Schema.VString) = varName ++ ".to_string()"+toClonedValueNonOptional varName t+ | isPassedByCopy t = varName+ | otherwise = varName ++ ".clone()"++isComplexForBTreeMap :: Schema.ConvexType -> Bool+isComplexForBTreeMap (Schema.VUnion _) = True+isComplexForBTreeMap (Schema.VReference _) = True+isComplexForBTreeMap Schema.VAny = True+isComplexForBTreeMap _ = False++toRustType :: String -> Schema.ConvexType -> (String, [String])+toRustType nameHint typ = case typ of+ Schema.VString -> ("String", [])+ Schema.VNumber -> ("f64", [])+ Schema.VInt64 -> ("i64", [])+ Schema.VFloat64 -> ("f64", [])+ Schema.VBoolean -> ("bool", [])+ Schema.VAny -> ("serde_json::Value", [])+ Schema.VNull -> ("()", [])+ Schema.VId t -> ("Id<types::" ++ toPascalCase t ++ "Doc>", [])+ Schema.VBytes -> ("Vec<u8>", [])+ Schema.VArray inner ->+ let (innerType, nested) = toRustType nameHint inner+ in ("Vec<" ++ innerType ++ ">", nested)+ Schema.VOptional inner ->+ let (innerType, nested) = toRustType nameHint inner+ in ("Option<" ++ innerType ++ ">", nested)+ Schema.VObject fields ->+ let className = toPascalCase nameHint ++ "Object"+ (fieldLines, nestedFields) = unzip $ map (generateField nameHint) (map (\(n, t) -> Schema.Field n t) fields)+ implBlock = generateToConvexValueImpl (className) fields+ fromBlock = generateFromConvexValueImpl (className) fields+ newModel =+ unlines+ [ indent 1 "#[derive(Default, Serialize, Deserialize, Debug, Clone, PartialEq)]",+ indent 1 ("pub struct " ++ className ++ " {"),+ unlines fieldLines,+ indent 1 "}",+ "",+ implBlock,+ "",+ fromBlock+ ]+ in ("types::" ++ className, concat nestedFields ++ [newModel])+ Schema.VUnion types ->+ let nonNullTypes = filter (/= Schema.VNull) types+ in case nonNullTypes of+ [] -> ("Option<()>", [])+ [singleType] ->+ let (innerType, nested) = toRustType nameHint singleType+ in ("Option<" ++ innerType ++ ">", nested)+ _ ->+ if all Schema.isLiteral nonNullTypes && not (null nonNullTypes)+ then+ let enumName = toPascalCase nameHint+ variantNames = map Schema.getLiteralString nonNullTypes+ buildVariantLines [] = []+ buildVariantLines (first : rest) =+ (indent 2 "#[default]\n" ++ indent 2 ("#[serde(rename = \"" ++ first ++ "\")]\n") ++ indent 2 (toPascalCase first ++ ","))+ : map (\v -> indent 2 ("#[serde(rename = \"" ++ v ++ "\")]\n") ++ indent 2 (toPascalCase v ++ ",")) rest+ newEnum =+ unlines+ [ indent 1 "#[derive(Default, Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]",+ indent 1 ("pub enum " ++ enumName ++ " {"),+ unlines $ buildVariantLines variantNames,+ indent 1 "}"+ ]+ in ("types::" ++ enumName, [newEnum])+ else ("serde_json::Value", []) -- Fallback for complex unions+ Schema.VLiteral _ -> (toPascalCase nameHint, [])+ Schema.VReference n -> ("types::" ++ toPascalCase n, [])+ Schema.VVoid -> ("()", [])++toRustBorrowType :: String -> String+toRustBorrowType rustType+ | rustType == "String" = "&str"+ | rustType == "serde_json::Value" = "&serde_json::Value"+ | rustType == "Vec<u8>" = "&[u8]"+ | "Id<" `isPrefixOf` rustType = "&" ++ rustType+ | "Vec<" `isPrefixOf` rustType = let inner = take (length rustType - 5) (drop 4 rustType) in "&[" ++ inner ++ "]"+ | "Option<" `isPrefixOf` rustType = let inner = take (length rustType - 8) (drop 7 rustType) in "Option<" ++ toRustBorrowType inner ++ ">"+ | otherwise = rustType++needsOptionalWrapper :: Schema.ConvexType -> Bool+needsOptionalWrapper (Schema.VOptional _) = True+needsOptionalWrapper (Schema.VUnion ts) = Schema.VNull `elem` ts+needsOptionalWrapper _ = False++toPascalCase :: String -> String+toPascalCase s = concatMap capitalize parts+ where+ parts = words $ map (\c -> if c == '_' then ' ' else c) s++capitalize :: String -> String+capitalize "" = ""+capitalize (c : cs) = toUpper c : cs++toSnakeCase :: String -> String+toSnakeCase "" = ""+toSnakeCase (c : cs) = toLower c : go cs+ where+ go (c' : cs')+ | isUpper c' = '_' : toLower c' : go cs'+ | otherwise = c' : go cs'+ go "" = ""++stripNewlines :: String -> String+stripNewlines s = unlines . filter (/= "") $ lines s
+ src/Backend/Rust/Validator.hs view
@@ -0,0 +1,107 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE InstanceSigs #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeSynonymInstances #-}++module Backend.Rust.Validator (run, RustValidator (..), RustValidatorEnv (..)) where++import Control.Monad.Reader+import Convex.Validator+import System.Directory (createDirectoryIfMissing, doesFileExist)+import System.Exit (ExitCode (..))+import System.FilePath ((</>))+import System.IO (hGetContents)+import System.Process (CreateProcess (..), StdStream (..), createProcess, proc, waitForProcess)++newtype RustValidator a = RustValidator+ { runRustValidator :: ReaderT RustValidatorEnv IO a+ }+ deriving (Functor, Applicative, Monad, MonadIO, MonadReader RustValidatorEnv, MonadFail)++data RustValidatorEnv = RustValidatorEnv {projectPath :: FilePath}++run :: RustValidatorEnv -> RustValidator a -> IO a+run renv action = runReaderT (runRustValidator action) renv++instance Validator RustValidator where+ -- \| Sets up the Rust validation sandbox project.+ setup = do+ config <- ask+ let rustProjectPath = projectPath config+ -- Create the directory structure.+ liftIO $ createDirectoryIfMissing True (rustProjectPath </> "src")++ -- Write the Cargo.toml file if it doesn't exist.+ let cargoTomlPath = rustProjectPath </> "Cargo.toml"+ cargoTomlExists <- liftIO $ doesFileExist cargoTomlPath+ if not cargoTomlExists+ then liftIO $ writeFile cargoTomlPath cargoTomlContent+ else return ()++ -- Write the src/lib.rs file if it doesn't exist.+ let libRsPath = rustProjectPath </> "src" </> "lib.rs"+ libRsExists <- liftIO $ doesFileExist libRsPath+ if not libRsExists+ then liftIO $ writeFile libRsPath libRsContent+ else return ()++ -- \| Validates the generated Rust code.+ validate generatedCode = do+ rustProjectPath <- asks projectPath+ let generatedFilePath = rustProjectPath </> "src" </> "generated_api.rs"++ -- Write the generated code to the sandbox project.+ liftIO $ writeFile generatedFilePath generatedCode++ -- Run `cargo fmt` to format the code.+ liftIO $ putStrLn "[Validator] Running 'cargo fmt'..."+ let fmtCmd = (proc "cargo" ["fmt"]) {cwd = Just rustProjectPath}+ (_, _, _, fmtHandle) <- liftIO $ createProcess fmtCmd+ fmtExitCode <- liftIO $ waitForProcess fmtHandle++ if fmtExitCode /= ExitSuccess+ then do+ liftIO $ putStrLn "[Validator] Error: 'cargo fmt' failed. The generated code has syntax errors."+ return Nothing+ else do+ -- Run `cargo check` to validate types and ownership.+ liftIO $ putStrLn "[Validator] Running 'cargo check'..."+ let checkCmd = (proc "cargo" ["check"]) {cwd = Just rustProjectPath, std_err = CreatePipe}+ (_, _, Just stdErr, checkHandle) <- liftIO $ createProcess checkCmd+ checkExitCode <- liftIO $ waitForProcess checkHandle++ if checkExitCode == ExitSuccess+ then do+ liftIO $ putStrLn "[Validator] Validation successful."+ -- Read file content from the checked and formatted file.+ content <- liftIO $ readFile generatedFilePath+ return $ Just content+ else do+ liftIO $ putStrLn "[Validator] Error: 'cargo check' failed. The generated code is invalid."+ liftIO $ hGetContents stdErr >>= putStrLn+ return Nothing++-- | The content for the validation project's Cargo.toml file.+cargoTomlContent :: String+cargoTomlContent =+ unlines+ [ "[package]",+ "name = \"validation-project\"",+ "version = \"0.1.0\"",+ "edition = \"2021\"",+ "",+ "[dependencies]",+ "convex = \"0.9.0\"",+ "serde = { version = \"1\", features = [\"derive\"] }",+ "serde_json = \"1\"",+ "thiserror = \"1.0\"",+ "anyhow = \"1.0\"",+ "futures-util = { version = \"0.3\" }",+ "tokio = { version = \"1\", features = [\"full\"] }"+ ]++-- | The content for the validation project's src/lib.rs file.+libRsContent :: String+libRsContent = "pub mod generated_api;\n"
+ src/Convex/Action/Parser.hs view
@@ -0,0 +1,244 @@+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}++module Convex.Action.Parser+ ( ConvexFunction (..),+ FuncType (..),+ parseActionFile,+ Schema.ConvexType (VVoid),+ )+where++import Control.Monad (void)+import qualified Convex.Schema.Parser as Schema+import Data.Functor (($>))+import Text.Parsec+import qualified Text.Parsec.Token as Token++type SchemaParser a = ParsecT String Schema.ParserState IO a++data FuncType = Query | Mutation | Action+ deriving (Show, Eq)++data ConvexFunction = ConvexFunction+ { funcName :: String,+ funcPath :: String,+ funcType :: FuncType,+ funcArgs :: [(String, Schema.ConvexType)],+ funcReturn :: Schema.ConvexType+ }+ deriving (Show, Eq)++-- Slightly different lexer for Actions.+langDef :: Token.GenLanguageDef String Schema.ParserState IO+langDef =+ Token.LanguageDef+ { Token.commentStart = "/*",+ Token.commentEnd = "*/",+ Token.commentLine = "//",+ Token.nestedComments = True,+ Token.identStart = letter <|> char '_',+ Token.identLetter = alphaNum <|> char '_',+ Token.opStart = oneOf ":!#$%&*+./<=>?@\\^|-~",+ Token.opLetter = oneOf ":!#$%&*+./<=>?@\\^|-~",+ Token.reservedOpNames = [],+ Token.reservedNames =+ [ "export",+ "declare",+ "const",+ "import",+ "from",+ "RegisteredQuery",+ "RegisteredMutation",+ "RegisteredAction",+ "Promise",+ "any",+ "string",+ "number",+ "boolean",+ "void",+ "GenericId",+ "DefaultFunctionArgs",+ "ArrayBuffer",+ "bigint"+ ],+ Token.caseSensitive = True+ }++lexer :: Token.GenTokenParser String Schema.ParserState IO+lexer = Token.makeTokenParser langDef++parens :: SchemaParser a -> SchemaParser a+parens = Token.parens lexer++whiteSpace :: SchemaParser ()+whiteSpace = Token.whiteSpace lexer++lexeme :: SchemaParser a -> SchemaParser a+lexeme = Token.lexeme lexer++identifier :: SchemaParser String+identifier = Token.identifier lexer++stringLiteral :: SchemaParser String+stringLiteral = Token.stringLiteral lexer++reserved :: String -> SchemaParser ()+reserved = Token.reserved lexer++braces :: SchemaParser a -> SchemaParser a+braces = Token.braces lexer++angles :: SchemaParser a -> SchemaParser a+angles p = lexeme (char '<') *> p <* lexeme (char '>')++dtsTypeParser :: SchemaParser Schema.ConvexType+dtsTypeParser = do+ -- A type can be a union of other types+ types <- sepBy1 singleType (lexeme (char '|'))+ let baseType = if length types == 1 then head types else Schema.VUnion types+ -- After parsing the base type, check for array suffixes `[]`+ arrayCount <- length <$> many (lexeme (string "[]"))+ -- Wrap the base type in VArray for each `[]` found+ return $ foldr (\_ acc -> Schema.VArray acc) baseType (replicate arrayCount ())+ where+ -- Parses both single identifiers (like `RoleEnum`)+ -- and qualified identifiers (like `Stripe.Subscription`).+ qualifiedIdentifierParser :: SchemaParser Schema.ConvexType+ qualifiedIdentifierParser = do+ parts <- sepBy1 identifier (lexeme (char '.'))+ if length parts > 1+ then -- If there's a dot, it's definitely an external type.+ return Schema.VAny+ else -- Otherwise, it's a single-word identifier, treat as a reference.+ return (Schema.VReference (head parts))++ singleType =+ (Schema.VString <$ try (reserved "string"))+ <|> (Schema.VNumber <$ try (reserved "number"))+ <|> (Schema.VBoolean <$ try (reserved "boolean"))+ <|> (Schema.VBytes <$ try (reserved "ArrayBuffer"))+ <|> (Schema.VInt64 <$ try (reserved "bigint"))+ <|> (Schema.VAny <$ try (reserved "any"))+ <|> (Schema.VLiteral <$> try stringLiteral)+ <|> (Schema.VId <$> try genericIdParser)+ <|> (Schema.VObject <$> try (braces (sepEndBy dtsFieldParser (lexeme (char ';')))))+ <|> try (parens dtsTypeParser)+ -- This is now the last option, which correctly handles all remaining identifiers.+ <|> qualifiedIdentifierParser++-- A parser for a single field inside an argument or object type+dtsFieldParser :: SchemaParser (String, Schema.ConvexType)+dtsFieldParser = lexeme $ do+ name <- identifier+ isOptional <- optionMaybe (lexeme (char '?'))+ void $ lexeme $ char ':'+ typ <- dtsTypeParser+ -- If the `?` was present, wrap the final type in VOptional+ let finalType = maybe typ (const $ Schema.VOptional typ) isOptional+ return (name, finalType)++-- A parser for `import("...").GenericId<"...">`+genericIdParser :: SchemaParser String+genericIdParser = do+ void $ reserved "import"+ void $ parens stringLiteral+ void $ lexeme $ char '.'+ void $ reserved "GenericId"+ angles stringLiteral++-- A parser for `import("...").DefaultFunctionArgs`+defaultFuncArgsParser :: SchemaParser ()+defaultFuncArgsParser = do+ void $ reserved "import"+ void $ parens stringLiteral+ void $ lexeme $ char '.'+ void $ reserved "DefaultFunctionArgs"++registeredFunctionParser :: String -> SchemaParser (Maybe ConvexFunction)+registeredFunctionParser fPath = lexeme $ do+ optional (try (lexeme (string "/**") *> manyTill anyChar (try (string "*/"))))+ whiteSpace++ reserved "export"+ reserved "declare"+ reserved "const"+ fName <- identifier+ void $ lexeme $ char ':'++ void $ reserved "import"+ void $ parens stringLiteral+ void $ lexeme $ char '.'++ fTypeStr <-+ choice+ [ try (reserved "RegisteredQuery" >> return "RegisteredQuery"),+ try (reserved "RegisteredMutation" >> return "RegisteredMutation"),+ try (reserved "RegisteredAction" >> return "RegisteredAction")+ ]++ let fType = case fTypeStr of+ "RegisteredQuery" -> Query+ "RegisteredMutation" -> Mutation+ "RegisteredAction" -> Action+ _ -> error "This case is unreachable due to the parser above"++ -- Parse the generic parameters+ (visibility, fArgs, fReturn) <- angles $ do+ vis <- stringLiteral+ void $ lexeme $ char ','+ args <-+ (try (braces (sepEndBy dtsFieldParser (lexeme (char ';')))))+ <|> (try defaultFuncArgsParser $> [])+ void $ lexeme $ char ','+ void $ reserved "Promise"+ ret <- angles ((reserved "void" $> Schema.VVoid) <|> dtsTypeParser)+ return (vis, args, ret)++ void $ lexeme $ char ';'++ case visibility of+ "public" -> return $ Just (ConvexFunction fName fPath fType fArgs fReturn)+ "internal" -> return Nothing+ other -> fail $ "Unknown or unhandled visibility in d.ts file: \"" ++ other ++ "\""++mapMaybe :: (a -> Maybe b) -> [a] -> [b]+mapMaybe _ [] = []+mapMaybe f (x : xs) =+ case f x of+ Just v -> v : mapMaybe f xs+ Nothing -> mapMaybe f xs++-- | A helper to parse and ignore statements that we don't care about.+ignoredStatementParser :: SchemaParser ()+ignoredStatementParser =+ choice . map try $+ [ importStatement,+ lineComment,+ blockComment,+ void (skipMany1 (oneOf " \t\n\r"))+ ]+ where+ importStatement =+ reserved "import"+ *> manyTill anyChar (char ';')+ *> pure ()+ lineComment =+ string (Token.commentLine langDef) *> manyTill anyChar (try (lookAhead (char '\n'))) *> pure ()+ blockComment =+ string (Token.commentStart langDef)+ *> manyTill anyChar (try (string (Token.commentEnd langDef)))+ *> pure ()++parseActionFile :: String -> SchemaParser [ConvexFunction]+parseActionFile path = do+ whiteSpace+ -- FIX: In a loop, consume either a function or an ignored statement,+ -- effectively skipping over comments and imports between functions.+ results <-+ many+ ( (try (Right <$> registeredFunctionParser path))+ <|> (try (Left <$> ignoredStatementParser))+ )+ -- Filter out the ignored statements (Lefts) and keep only the functions (Rights).+ return $ mapMaybe (either (const Nothing) id) results
+ src/Convex/Parser.hs view
@@ -0,0 +1,146 @@+{-# LANGUAGE OverloadedStrings #-}++module Convex.Parser (parseProject, ParsedProject (..), apiFileParser) where++import Control.Monad (forM, void)+import qualified Convex.Action.Parser as Action+import qualified Convex.Schema.Parser as Schema+import Data.List (sort)+import qualified Data.Map as Map+import System.FilePath (replaceExtension, (</>))+import Text.Parsec+import Text.Parsec.Language (emptyDef)+import qualified Text.Parsec.Token as Token++data ParsedProject = ParsedProject+ { ppSchema :: Schema.Schema,+ ppConstants :: Map.Map String Schema.ConvexType,+ ppFunctions :: [Action.ConvexFunction]+ }+ deriving (Show, Eq)++apiLexer :: Token.TokenParser ()+apiLexer =+ Token.makeTokenParser+ emptyDef+ { Token.identStart = letter <|> char '_',+ Token.identLetter = alphaNum <|> char '_',+ Token.reservedNames = ["typeof", "declare", "const", "ApiFromModules"]+ }++apiStringLiteral :: Parsec String () String+apiStringLiteral = Token.stringLiteral apiLexer++apiIdentifier :: Parsec String () String+apiIdentifier = Token.identifier apiLexer++apiReserved :: String -> Parsec String () ()+apiReserved = Token.reserved apiLexer++apiFileParser :: Parsec String () [String] -- We only need the paths+apiFileParser = do+ -- Find the start of the object we care about+ void $ manyTill anyChar (try (string "declare const fullApi: ApiFromModules<"))+ -- Enter the braces of the object+ paths <- Token.braces apiLexer (sepEndBy singlePath (Token.lexeme apiLexer (char ';')))+ return paths+ where+ -- This parses a single line like: "admin/actions": typeof admin_actions+ singlePath = do+ path <- apiStringLiteral <|> apiIdentifier+ void $ Token.lexeme apiLexer $ char ':'+ void $ apiReserved "typeof"+ void $ apiIdentifier -- Consume the alias, we don't need it+ return path++parseProject :: FilePath -> FilePath -> IO (Either String ParsedProject)+parseProject schemaPath declRootDir = do+ -- Parse the source schema file first to get tables and the initial state with constants.+ schemaContent <- readFile schemaPath+ schemaResult <- Schema.parseSchema schemaContent++ case schemaResult of+ Left err -> return $ Left ("Failed to parse schema.ts: " ++ show err)+ Right schemaFile -> do+ -- Re-construct the initial state for the action parser from the parsed constants.+ let initialState = Schema.ParserState {Schema.psConstants = Schema.parsedConstants schemaFile}++ -- Parse the _generated/api.d.ts file to discover function modules.+ let apiFilePath = declRootDir </> "_generated" </> "api.d.ts"+ apiFileContent <- readFile apiFilePath+ let modulePaths = case parse apiFileParser apiFilePath apiFileContent of+ Left _ -> []+ Right paths -> filter (/= "schema") paths++ putStrLn $ "Found " ++ show (length modulePaths) ++ " action modules in: " ++ apiFilePath++ -- For each discovered module, parse its corresponding .d.ts file.+ allFunctions <- fmap concat $ forM modulePaths $ \modulePath -> do+ let fullPath = declRootDir </> replaceExtension modulePath ".d.ts"+ let astPath = replaceExtension modulePath ""++ actionContent <- readFile fullPath+ actionResult <- runParserT (Action.parseActionFile astPath) initialState fullPath actionContent++ case actionResult of+ Left err -> do+ putStrLn $ "Failed to parse actions from: " ++ fullPath ++ " | Error: " ++ show err+ return []+ Right funcs -> do+ putStrLn $ "Parsed actions from: " ++ fullPath+ putStrLn $ show (length funcs) ++ " functions found"+ return funcs++ let project =+ ParsedProject+ { ppSchema = Schema.parsedSchema schemaFile,+ ppConstants = Schema.parsedConstants schemaFile,+ ppFunctions = allFunctions+ }++ return $ Right . unifyProjectTypes $ project++type UnionSignatureMap = Map.Map [String] String++-- | Pre-processes the parsed project to replace anonymous unions with named references+-- if they structurally match.+unifyProjectTypes :: ParsedProject -> ParsedProject+unifyProjectTypes project =+ let unionMap = buildUnionSignatureMap (ppConstants project)+ in project {ppFunctions = map (unifyFunctionTypes unionMap) (ppFunctions project)}+ where+ buildUnionSignatureMap :: Map.Map String Schema.ConvexType -> UnionSignatureMap+ buildUnionSignatureMap constants =+ Map.fromList+ [ (sort $ map Schema.getLiteralString literals, name)+ | (name, Schema.VUnion literals) <- Map.toList constants,+ all Schema.isLiteral literals+ ]++ unifyFunctionTypes :: UnionSignatureMap -> Action.ConvexFunction -> Action.ConvexFunction+ unifyFunctionTypes unionMap func =+ func+ { Action.funcArgs = map (unifyArgType unionMap) (Action.funcArgs func),+ Action.funcReturn = unifyTypeRecursively unionMap (Action.funcReturn func)+ }++ unifyArgType :: UnionSignatureMap -> (String, Schema.ConvexType) -> (String, Schema.ConvexType)+ unifyArgType unionMap (argName, argType) =+ (argName, unifyTypeRecursively unionMap argType)++ -- This new recursive function traverses the entire type structure.+ unifyTypeRecursively :: UnionSignatureMap -> Schema.ConvexType -> Schema.ConvexType+ unifyTypeRecursively unionMap u@(Schema.VUnion literals)+ | all Schema.isLiteral literals =+ let signature = sort $ map Schema.getLiteralString literals+ in case Map.lookup signature unionMap of+ Just refName -> Schema.VReference refName+ Nothing -> u+ | otherwise = u+ unifyTypeRecursively unionMap (Schema.VObject fields) =+ Schema.VObject $ map (\(name, t) -> (name, unifyTypeRecursively unionMap t)) fields+ unifyTypeRecursively unionMap (Schema.VArray inner) =+ Schema.VArray $ unifyTypeRecursively unionMap inner+ unifyTypeRecursively unionMap (Schema.VOptional inner) =+ Schema.VOptional $ unifyTypeRecursively unionMap inner+ unifyTypeRecursively _ otherType = otherType -- Base cases
+ src/Convex/Schema/Parser.hs view
@@ -0,0 +1,282 @@+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}++module Convex.Schema.Parser+ ( parseSchema,+ ParsedFile (..),+ Schema (..),+ Table (..),+ Index (..),+ Field (..),+ ConvexType (..),+ ParserState (..),+ initialState,+ getLiteralString,+ isLiteral,+ )+where++import Control.Monad+import Data.Map (Map)+import qualified Data.Map as Map+import Text.Parsec+import qualified Text.Parsec.Language as Token+import qualified Text.Parsec.Token as Token++type SchemaParser a = ParsecT String ParserState IO a++data ParserState = ParserState+ { psConstants :: Map String ConvexType+ }+ deriving (Show, Eq)++initialState :: ParserState+initialState = ParserState {psConstants = Map.empty}++data ParsedFile = ParsedFile+ { parsedConstants :: Map String ConvexType,+ parsedSchema :: Schema+ }+ deriving (Show, Eq)++newtype Schema = Schema {getTables :: [Table]}+ deriving (Show, Eq)++data Index = Index+ { indexName :: String,+ indexFields :: [String]+ }+ deriving (Show, Eq)++data Table = Table+ { tableName :: String,+ tableFields :: [Field],+ tableIndexes :: [Index]+ }+ deriving (Show, Eq)++data Field = Field+ { fieldName :: String,+ fieldType :: ConvexType+ }+ deriving (Show, Eq)++data ConvexType+ = VString+ | VNumber+ | VInt64+ | VFloat64+ | VBoolean+ | VBytes+ | VNull+ | VAny+ | VId String+ | VArray ConvexType+ | VObject [(String, ConvexType)]+ | VOptional ConvexType+ | VUnion [ConvexType]+ | VLiteral String+ | VReference String+ | VVoid+ deriving (Show, Eq, Ord)++getLiteralString :: ConvexType -> String+getLiteralString (VLiteral str) = str+getLiteralString _ = error "Expected a literal type"++isLiteral :: ConvexType -> Bool+isLiteral (VLiteral _) = True+isLiteral _ = False++langDef :: Token.GenLanguageDef String ParserState IO+langDef =+ Token.LanguageDef+ { Token.commentStart = "/*",+ Token.nestedComments = True,+ Token.commentEnd = "*/",+ Token.commentLine = "//",+ Token.opStart = oneOf ":!#$%&*+./<=>?@\\^|-~",+ Token.opLetter = oneOf ":!#$%&*+./<=>?@\\^|-~",+ Token.reservedOpNames = [],+ Token.identStart = letter <|> char '_',+ Token.identLetter = alphaNum <|> char '_',+ Token.reservedNames =+ [ "defineSchema",+ "defineSchema(",+ "defineTable",+ "v",+ "export",+ "default",+ "import",+ "from",+ "const",+ "type",+ "keyof",+ "typeof"+ ],+ Token.caseSensitive = True+ }++lexer :: Token.GenTokenParser String ParserState IO+lexer = Token.makeTokenParser langDef++whiteSpace :: SchemaParser ()+whiteSpace = Token.whiteSpace lexer++lexeme :: SchemaParser a -> SchemaParser a+lexeme = Token.lexeme lexer++identifier :: SchemaParser String+identifier = Token.identifier lexer++stringLiteral :: SchemaParser String+stringLiteral = Token.stringLiteral lexer++reserved :: String -> SchemaParser ()+reserved = Token.reserved lexer++parens :: SchemaParser a -> SchemaParser a+parens = Token.parens lexer++braces :: SchemaParser a -> SchemaParser a+braces = Token.braces lexer++brackets :: SchemaParser a -> SchemaParser a+brackets = Token.brackets lexer++topLevelStatementEnd :: SchemaParser ()+topLevelStatementEnd = void (optional (lexeme (char ';'))) *> whiteSpace++itemEnd :: SchemaParser ()+itemEnd = do+ optional (lexeme (char ','))+ optional (lexeme (char ';'))+ whiteSpace++fieldToTuple :: Field -> (String, ConvexType)+fieldToTuple (Field name typ) = (name, typ)++fieldParser :: SchemaParser Field+fieldParser = lexeme $ do+ key <- identifier <|> stringLiteral+ void $ lexeme $ char ':'+ value <- convexTypeParser+ return $ Field key value++indexParser :: SchemaParser Index+indexParser = lexeme $ do+ void $ char '.'+ reserved "index"+ (iName, iFields) <- parens $ do+ name <- stringLiteral+ void $ lexeme $ char ','+ fields <- brackets $ sepEndBy stringLiteral (lexeme $ char ',')+ return (name, fields)+ return $ Index iName iFields++tableParser :: SchemaParser Table+tableParser = lexeme $ do+ tName <- identifier <|> stringLiteral+ void $ lexeme $ char ':'+ reserved "defineTable"+ -- First, parse the table definition itself inside the parentheses.+ fields <- parens $ do+ tableDef <- (try (VObject . map fieldToTuple <$> braces (sepEndBy fieldParser (lexeme $ char ',')))) <|> (VReference <$> identifier)+ case tableDef of+ VReference refName -> do+ st <- getState+ case Map.lookup refName (psConstants st) of+ Just (VObject fs) -> return $ map (\(n, t) -> Field n t) fs+ _ -> fail $ "Table '" ++ tName ++ "' references an unknown or non-object constant: " ++ refName+ VObject fs -> return $ map (\(n, t) -> Field n t) fs+ _ -> fail "Invalid table definition: expected an object or a reference."++ -- After parsing defineTable(...), now look for zero or more chained .index() calls.+ indexes <- many indexParser++ itemEnd+ return $ Table tName fields indexes++structParser :: SchemaParser ConvexType+structParser = do+ res <- VObject . map fieldToTuple <$> braces (sepEndBy fieldParser (lexeme $ char ','))+ itemEnd+ return res++convexTypeParser :: SchemaParser ConvexType+convexTypeParser =+ choice . map try $+ [ vParser,+ structParser,+ referenceParser+ ]+ where+ vParser = do+ void $ lexeme $ reserved "v"+ void $ lexeme $ char '.'+ typeName <- identifier+ case typeName of+ "string" -> VString <$ parens (return ())+ "number" -> VNumber <$ parens (return ())+ "boolean" -> VBoolean <$ parens (return ())+ "bytes" -> VBytes <$ parens (return ())+ "int64" -> VInt64 <$ parens (return ())+ "float64" -> VFloat64 <$ parens (return ())+ "null" -> VNull <$ parens (return ())+ "any" -> VAny <$ parens (return ())+ "id" -> VId <$> parens stringLiteral+ "array" -> VArray <$> parens convexTypeParser+ "object" -> parens structParser+ "optional" -> VOptional <$> parens convexTypeParser+ "union" -> VUnion <$> parens (sepBy convexTypeParser (lexeme $ char ','))+ "literal" -> VLiteral <$> parens stringLiteral+ _ -> fail $ "Unknown v-dot type: " ++ typeName+ referenceParser = VReference <$> identifier++topLevelConstParser :: SchemaParser ()+topLevelConstParser = lexeme $ do+ void $ optional (reserved "export")+ reserved "const"+ constName <- identifier+ void $ lexeme $ char '='+ constType <-+ try (reserved "defineTable" *> (VObject . map fieldToTuple <$> parens (braces (many fieldParser))))+ <|> convexTypeParser+ topLevelStatementEnd+ modifyState (\s -> s {psConstants = Map.insert constName constType (psConstants s)})++topLevelTypeParser :: SchemaParser ()+topLevelTypeParser = lexeme $ do+ void $ optional (reserved "export")+ reserved "type"+ typeName <- identifier+ void $ lexeme $ char '='+ optional (reserved "typeof")+ refType <- convexTypeParser+ topLevelStatementEnd+ modifyState (\s -> s {psConstants = Map.insert typeName refType (psConstants s)})++parseSchema :: String -> IO (Either ParseError ParsedFile)+parseSchema input = do+ -- First Pass: Collect all top-level definitions (consts and types).+ let definitionsPassParser = many (try topLevelConstParser <|> try topLevelTypeParser <|> (anyChar >> return ()))+ constsState <- execParser (definitionsPassParser *> getState) initialState "(schema.ts)" input++ -- Second Pass: Parse the schema, using the constants we just found.+ let schemaPassParser = do+ _ <- manyTill anyChar (lookAhead (try (reserved "defineSchema(")))+ reserved "defineSchema"+ tables <- parens $ braces $ many tableParser+ return $ Schema tables++ schemaResult <- execParser schemaPassParser constsState "(schema.ts)" input++ return $ Right (ParsedFile (psConstants constsState) schemaResult)++-- | A helper to run a parser and return the result, simplifying error handling.+execParser :: SchemaParser a -> ParserState -> SourceName -> String -> IO a+execParser p st name input = do+ result <- runParserT p st name input+ case result of+ Left err -> fail (show err)+ Right res -> return res
+ src/Convex/Validator.hs view
@@ -0,0 +1,22 @@+module Convex.Validator+ ( Validator (..),+ ValidationConfig (..),+ )+where++-- | Configuration for the validation setup process.+newtype ValidationConfig = ValidationConfig+ { -- | The root directory where validator projects will be created.+ -- e.g., "~/.config/convex-schema-parser"+ validationDir :: FilePath+ }++-- | A typeclass for language-specific code validators.+class Validator v where+ -- | Sets up the validation environment (e.g., creates a sandbox project)+ -- and returns a handle to it. This function is idempotent.+ setup :: v ()++ -- | Validates a string of generated code using the provided handle.+ -- Returns the potentially formatted and checked code.+ validate :: String -> v (Maybe String)
+ src/PathTree.hs view
@@ -0,0 +1,27 @@+module PathTree+ ( PathTree (..),+ buildPathTree,+ )+where++import qualified Convex.Action.Parser as Action+import Data.List.Split (splitOn)+import qualified Data.Map.Strict as Map++data PathTree = FuncNode Action.ConvexFunction | DirNode (Map.Map String PathTree) deriving (Show)++buildPathTree :: [Action.ConvexFunction] -> PathTree+buildPathTree = foldl (flip insertFunc) (DirNode Map.empty)+ where+ insertFunc :: Action.ConvexFunction -> PathTree -> PathTree+ insertFunc func (DirNode dir) = DirNode (go (splitOn "/" (Action.funcPath func)) func dir)+ insertFunc _ node = node++ go :: [String] -> Action.ConvexFunction -> Map.Map String PathTree -> Map.Map String PathTree+ go [] func dir = Map.insert (Action.funcName func) (FuncNode func) dir+ go (p : ps) func dir =+ let subTree = Map.findWithDefault (DirNode Map.empty) p dir+ newSubTree = case subTree of+ DirNode subDirMap -> DirNode (go ps func subDirMap)+ FuncNode _ -> error $ "Path conflict: cannot create submodule in path containing function: " ++ p+ in Map.insert p newSubTree dir
+ test/ActionParserTest.hs view
@@ -0,0 +1,208 @@+{-# LANGUAGE OverloadedStrings #-}++module ActionParserTest (tests) where++import qualified Convex.Action.Parser as Action+import qualified Convex.Schema.Parser as Schema+import Test.HUnit+import Text.Parsec (runParserT)++tests :: Test+tests =+ "Action Definition Parser"+ ~: test+ [ runActionTest "parses createProject mutation" "test/api" sampleCreateProject expectedCreateProject,+ runActionTest "ignores internal actions" "test/api" sampleInternalAction [],+ runActionTest "parses public query with complex return" "functions/users" samplePublicGetOrgs expectedPublicGetOrgs,+ runActionTest "parses public action with optional args" "functions/stripe" samplePublicAction expectedPublicAction,+ runActionTest "parses internal action with external type as VAny" "functions/stripe" sampleStripeSubscriptionAction expectedStripeSubscriptionAction,+ runActionTest "parses public action with external type" "functions/stripe" sampleStripeCheckoutAction sampleStripeCheckoutActionExpected,+ runActionTest "parses public action with external type as VAny" "functions/stripe" sampleStripeSubscriptionActionPublic expectedStripeSubscriptionActionPublic,+ runActionTest "parses createAsset mutation with complex args" "functions/assets" sampleCreateAssetAction expectedCreateAssetAction+ ]+ where+ runActionTest testName path input expected =+ testName ~: TestCase $ do+ result <- runParserT (Action.parseActionFile path) Schema.initialState "(test)" input+ case result of+ Left err -> assertFailure ("Parser failed: " ++ show err)+ Right funcs -> funcs @?= expected++sampleCreateProject :: String+sampleCreateProject =+ unlines+ [ "export declare const createProject: import(\"convex/server\").RegisteredMutation<\"public\", {",+ " project_name?: string;",+ " tenant_id: import(\"convex/values\").GenericId<\"tenants\">;",+ "}, Promise<import(\"convex/values\").GenericId<\"projects\">>>;"+ ]++expectedCreateProject :: [Action.ConvexFunction]+expectedCreateProject =+ [ Action.ConvexFunction+ { Action.funcName = "createProject",+ Action.funcPath = "test/api",+ Action.funcType = Action.Mutation,+ Action.funcArgs =+ [ ("project_name", Schema.VOptional Schema.VString),+ ("tenant_id", Schema.VId "tenants")+ ],+ Action.funcReturn = Schema.VId "projects"+ }+ ]++sampleInternalAction :: String+sampleInternalAction = "export declare const sendOtp: import(\"convex/server\").RegisteredAction<\"internal\", { email: string; }, Promise<{ success: boolean; }>>;"++samplePublicGetOrgs :: String+samplePublicGetOrgs =+ unlines+ [ "export declare const getOrgs: import(\"convex/server\").RegisteredQuery<\"public\", {}, Promise<{",+ " name: string;",+ " _id: import(\"convex/values\").GenericId<\"memberships\">;",+ " _creationTime: number;",+ " user_id: import(\"convex/values\").GenericId<\"users\">;",+ " tenant_id: import(\"convex/values\").GenericId<\"tenants\">;",+ " roles: (\"viewer\" | \"user\" | \"admin\")[];",+ "}[]>>;"+ ]++expectedPublicGetOrgs :: [Action.ConvexFunction]+expectedPublicGetOrgs =+ [ Action.ConvexFunction+ { Action.funcName = "getOrgs",+ Action.funcPath = "functions/users",+ Action.funcType = Action.Query,+ Action.funcArgs = [],+ Action.funcReturn =+ Schema.VArray+ ( Schema.VObject+ [ ("name", Schema.VString),+ ("_id", Schema.VId "memberships"),+ ("_creationTime", Schema.VNumber),+ ("user_id", Schema.VId "users"),+ ("tenant_id", Schema.VId "tenants"),+ ( "roles",+ Schema.VArray+ ( Schema.VUnion+ [ Schema.VLiteral "viewer",+ Schema.VLiteral "user",+ Schema.VLiteral "admin"+ ]+ )+ )+ ]+ )+ }+ ]++samplePublicAction :: String+samplePublicAction =+ unlines+ [ "export declare const createCheckoutSession: import(\"convex/server\").RegisteredAction<\"public\", {",+ " next_url?: string;",+ " tenant_id: import(\"convex/values\").GenericId<\"tenants\">;",+ " price_id: string;",+ "}, Promise<string>>;"+ ]++expectedPublicAction :: [Action.ConvexFunction]+expectedPublicAction =+ [ Action.ConvexFunction+ { Action.funcName = "createCheckoutSession",+ Action.funcPath = "functions/stripe",+ Action.funcType = Action.Action,+ Action.funcArgs =+ [ ("next_url", Schema.VOptional Schema.VString),+ ("tenant_id", Schema.VId "tenants"),+ ("price_id", Schema.VString)+ ],+ Action.funcReturn = Schema.VString+ }+ ]++sampleStripeSubscriptionAction :: String+sampleStripeSubscriptionAction = "export declare const stripeSubscription: import(\"convex/server\").RegisteredAction<\"internal\", { subscription_id: string; }, Promise<Stripe.Subscription>>;"++expectedStripeSubscriptionAction :: [Action.ConvexFunction]+expectedStripeSubscriptionAction = []++sampleStripeSubscriptionActionPublic :: String+sampleStripeSubscriptionActionPublic =+ unlines+ [ "export declare const stripeSubscription: import(\"convex/server\").RegisteredAction<\"public\", {",+ " subscription_id: string;",+ "}, Promise<Stripe.Subscription>>;"+ ]++-- Even though the return type is an external type, it should still parse correctly.+expectedStripeSubscriptionActionPublic :: [Action.ConvexFunction]+expectedStripeSubscriptionActionPublic =+ [ Action.ConvexFunction+ { Action.funcName = "stripeSubscription",+ Action.funcPath = "functions/stripe",+ Action.funcType = Action.Action,+ Action.funcArgs = [("subscription_id", Schema.VString)],+ Action.funcReturn = Schema.VAny -- External type, parsed as VAny+ }+ ]++sampleStripeCheckoutAction :: String+sampleStripeCheckoutAction =+ unlines+ [ "export declare const createCheckoutSession: import(\"convex/server\").RegisteredAction<\"public\", {",+ " next_url?: string;",+ " tenant_id: import(\"convex/values\").GenericId<\"tenants\">;",+ " price_id: string;",+ "}, Promise<string>>;"+ ]++sampleStripeCheckoutActionExpected :: [Action.ConvexFunction]+sampleStripeCheckoutActionExpected =+ [ Action.ConvexFunction+ { Action.funcName = "createCheckoutSession",+ Action.funcPath = "functions/stripe",+ Action.funcType = Action.Action,+ Action.funcArgs =+ [ ("next_url", Schema.VOptional Schema.VString),+ ("tenant_id", Schema.VId "tenants"),+ ("price_id", Schema.VString)+ ],+ Action.funcReturn = Schema.VString+ }+ ]++sampleCreateAssetAction :: String+sampleCreateAssetAction =+ unlines+ [ "export declare const createAsset: import(\"convex/server\").RegisteredMutation<\"public\", {",+ " projectId: import(\"convex/values\").GenericId<\"projects\">;",+ " linkMetadata: {",+ " summary: ArrayBuffer;",+ " sample_rate: number;",+ " length: number;",+ " linkable_uri: string;",+ " };",+ "}, Promise<import(\"convex/values\").GenericId<\"assets\">>>;"+ ]++expectedCreateAssetAction :: [Action.ConvexFunction]+expectedCreateAssetAction =+ [ Action.ConvexFunction+ { Action.funcName = "createAsset",+ Action.funcPath = "functions/assets",+ Action.funcType = Action.Mutation,+ Action.funcArgs =+ [ ("projectId", Schema.VId "projects"),+ ( "linkMetadata",+ Schema.VObject+ [ ("summary", Schema.VBytes),+ ("sample_rate", Schema.VNumber),+ ("length", Schema.VNumber),+ ("linkable_uri", Schema.VString)+ ]+ )+ ],+ Action.funcReturn = Schema.VId "assets"+ }+ ]
+ test/ApiParserTest.hs view
@@ -0,0 +1,48 @@+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}++module ApiParserTest (tests) where++-- Import the module and function we are testing+import qualified Convex.Parser as APIParser+import Test.HUnit+import Text.Parsec (parse)++-- The test suite for the apiFileParser, exported to be used by Main.+tests :: Test+tests =+ "API File Parser"+ ~: runTest+ "parses module paths from api.d.ts"+ (APIParser.apiFileParser)+ sampleApiFile+ expectedApiPaths+ where+ runTest testName parser input expected =+ testName ~: TestCase $ do+ -- Use the pure `parse` function since this parser is stateless.+ case parse parser "(test input)" input of+ Left err -> assertFailure ("Parser failed: " ++ show err)+ Right actual -> actual @?= expected++-- A sample input string that mimics the structure of a real api.d.ts file.+sampleApiFile :: String+sampleApiFile =+ unlines+ [ "/* eslint-disable */",+ "import type * as admin_actions from \"../admin/actions.js\";",+ "import type * as custom from \"../custom.js\";",+ "declare const fullApi: ApiFromModules<{",+ " \"admin/actions\": typeof admin_actions;",+ " custom: typeof custom;",+ " \"stripe/checkout\": typeof stripe_checkout;",+ "}>;"+ ]++-- The list of module paths we expect the parser to extract.+expectedApiPaths :: [String]+expectedApiPaths =+ [ "admin/actions",+ "custom",+ "stripe/checkout"+ ]
+ test/Main.hs view
@@ -0,0 +1,25 @@+module Main (main) where++import qualified ActionParserTest+import qualified ApiParserTest+import qualified RustSerializationTest+import qualified SchemaParserTest+import System.Exit (exitFailure, exitSuccess)+import Test.HUnit++main :: IO ()+main = do+ -- Combine all test suites into a single list+ let allTests =+ TestList+ [ ApiParserTest.tests,+ ActionParserTest.tests,+ SchemaParserTest.tests,+ RustSerializationTest.tests+ ]++ -- Run the combined test suite+ counts <- runTestTT allTests+ if errors counts + failures counts == 0+ then exitSuccess+ else exitFailure
+ test/RustSerializationTest.hs view
@@ -0,0 +1,195 @@+{-# LANGUAGE OverloadedStrings #-}++module RustSerializationTest (tests) where++import qualified Backend.Rust as Rust+import qualified Convex.Action.Parser as Action+import qualified Convex.Parser as Convex+import qualified Convex.Schema.Parser as Schema+import Data.List (isPrefixOf, tails)+import qualified Data.Map as Map+import Test.HUnit+import Text.Parsec (runParserT)++-- Helper to run the test+runSerializationTest :: String -> Action.ConvexFunction -> String -> Test+runSerializationTest testName func expected =+ testName ~: TestCase $ do+ -- We create a dummy project containing only our test function.+ let project =+ Convex.ParsedProject+ { Convex.ppConstants = Map.empty,+ Convex.ppSchema = Schema.Schema {Schema.getTables = []},+ Convex.ppFunctions = [func]+ }+ -- Generate the Rust code.+ let generatedCode = Rust.generateRustCode project+ -- For simplicity in this test, we are just checking if the expected output+ -- is contained within the generated code.+ assertBool ("Generated code does not contain the expected implementation for to_convex_value:\n\nExpected to find:\n" ++ expected ++ "\n\nIn generated code:\n" ++ generatedCode) (normalizeString expected `isInfixOf` normalizeString generatedCode)++-- | Removes all whitespace/newlines etc. from the string.+normalizeString :: String -> String+normalizeString = filter (/= '\n') . filter (/= ' ') . filter (/= '\t')++-- Test Input: A complex nested object definition+complexCreateFunction :: Action.ConvexFunction+complexCreateFunction =+ Action.ConvexFunction+ { Action.funcName = "createComplexDoc",+ Action.funcPath = "documents",+ Action.funcType = Action.Mutation,+ Action.funcArgs =+ [ ( "payload",+ Schema.VObject+ [ ("name", Schema.VOptional Schema.VString),+ ("value", Schema.VInt64),+ ( "tags",+ Schema.VOptional+ ( Schema.VArray+ ( Schema.VObject+ [ ("tag_name", Schema.VString),+ ("tag_type", Schema.VReference "TagTypeEnum")+ ]+ )+ )+ )+ ]+ )+ ],+ Action.funcReturn = Schema.VId "documents"+ }++-- The exact, correct Rust code we expect the generator to produce.+expectedComplexSerializationImpl :: String+expectedComplexSerializationImpl =+ unlines+ [ "impl CreateComplexDocPayloadObject {",+ " pub fn to_convex_value(&self) -> Result<Value, ApiError> {",+ " let mut btmap = BTreeMap::new();",+ " if let Some(v) = &self.name {",+ " btmap.insert(\"name\".to_string(), Value::from(v.to_string()));",+ " }",+ " btmap.insert(\"value\".to_string(), Value::from(self.value));",+ " if let Some(v) = &self.tags {",+ " btmap.insert(\"tags\".to_string(), Value::Array(v.iter().map(|item| item.to_convex_value()).collect::<Result<Vec<_>, _>>()?));",+ " }",+ " Ok(Value::Object(btmap))",+ " }",+ "}"+ ]++getAssetsFunction :: Action.ConvexFunction+getAssetsFunction =+ Action.ConvexFunction+ { Action.funcName = "getAssets",+ Action.funcPath = "functions/assets",+ Action.funcType = Action.Query,+ Action.funcArgs = [("projectId", Schema.VId "projects")],+ Action.funcReturn =+ Schema.VArray+ ( Schema.VObject+ [ ("_id", Schema.VId "assets"),+ ("_creationTime", Schema.VNumber),+ ("project_id", Schema.VId "projects"),+ ("asset_name", Schema.VString),+ ( "link_metadata",+ Schema.VObject+ [ ("summary", Schema.VBytes),+ ("sample_rate", Schema.VInt64),+ ("length", Schema.VInt64)+ ]+ ),+ ("asset_essence_mtime", Schema.VInt64)+ ]+ )+ }++expectedGetAssetsSerializationImpl :: String+expectedGetAssetsSerializationImpl =+ unlines+ [ "impl TryFrom<Value> for GetAssetsReturnObject {",+ " type Error = ApiError;",+ " fn try_from(value: Value) -> Result<Self, Self::Error> {",+ " let obj = match value {",+ " Value::Object(map) => map,",+ " _ => return Err(ApiError::ConvexClientError(\"Expected object\".to_string())),",+ " };",+ " fn get__id(map: &BTreeMap<String, Value>, key: &str) -> Result<Id<types::AssetsDoc>, ApiError> {",+ " match map.get(key) {",+ " Some(v) => {",+ " Ok(FromConvexValue::from_convex(v.clone())?)",+ " }",+ " _ => return Err(ApiError::ConvexClientError(format!(\"Expected field (Id<types::AssetsDoc>) '{}' not found\", key))),",+ " }",+ " }",+ " fn get__creation_time(map: &BTreeMap<String, Value>, key: &str) -> Result<f64, ApiError> {",+ " match map.get(key) {",+ " Some(v) => {",+ " Ok(FromConvexValue::from_convex(v.clone())?)",+ " }",+ " _ => return Err(ApiError::ConvexClientError(format!(\"Expected field (f64) '{}' not found\", key))),",+ " }",+ " }",+ " fn get_project_id(map: &BTreeMap<String, Value>, key: &str) -> Result<Id<types::ProjectsDoc>, ApiError> {",+ " match map.get(key) {",+ " Some(v) => {",+ " Ok(FromConvexValue::from_convex(v.clone())?)",+ " }",+ " _ => return Err(ApiError::ConvexClientError(format!(\"Expected field (Id<types::ProjectsDoc>) '{}' not found\", key))),",+ " }",+ " }",+ " fn get_asset_name(map: &BTreeMap<String, Value>, key: &str) -> Result<String, ApiError> {",+ " match map.get(key) {",+ " Some(v) => {",+ " Ok(FromConvexValue::from_convex(v.clone())?)",+ " }",+ " _ => return Err(ApiError::ConvexClientError(format!(\"Expected field (String) '{}' not found\", key))),",+ " }",+ " }",+ " fn get_link_metadata(map: &BTreeMap<String, Value>, key: &str) -> Result<types::GetAssetsReturnLinkMetadataObject, ApiError> {",+ " match map.get(key) {",+ " Some(v) => {",+ " Ok(FromConvexValue::from_convex(v.clone())?)",+ " }",+ " _ => return Err(ApiError::ConvexClientError(format!(\"Expected field (types::GetAssetsReturnLinkMetadataObject) '{}' not found\", key))),",+ " }",+ " }",+ " fn get_asset_essence_mtime(map: &BTreeMap<String, Value>, key: &str) -> Result<i64, ApiError> {",+ " match map.get(key) {",+ " Some(v) => {",+ " Ok(FromConvexValue::from_convex(v.clone())?)",+ " }",+ " _ => return Err(ApiError::ConvexClientError(format!(\"Expected field (i64) '{}' not found\", key))),",+ " }",+ " }",+ " Ok(GetAssetsReturnObject {",+ " _id: get__id(&obj, \"_id\")?,",+ " _creation_time: get__creation_time(&obj, \"_creationTime\")?,",+ " project_id: get_project_id(&obj, \"project_id\")?,",+ " asset_name: get_asset_name(&obj, \"asset_name\")?,",+ " link_metadata: get_link_metadata(&obj, \"link_metadata\")?,",+ " asset_essence_mtime: get_asset_essence_mtime(&obj, \"asset_essence_mtime\")?",+ " })",+ " }",+ "}"+ ]++-- Main test suite+tests :: Test+tests =+ "Rust Serialization"+ ~: test+ [ runSerializationTest+ "generates correct to_convex_value for complex nested objects"+ complexCreateFunction+ expectedComplexSerializationImpl,+ runSerializationTest+ "generates correct from_convex for getAssets function"+ getAssetsFunction+ expectedGetAssetsSerializationImpl+ ]++-- Helper to check for substring containment.+isInfixOf :: (Eq a) => [a] -> [a] -> Bool+isInfixOf needle haystack = any (isPrefixOf needle) (tails haystack)
+ test/SchemaParserTest.hs view
@@ -0,0 +1,302 @@+{-# LANGUAGE OverloadedStrings #-}++module SchemaParserTest (tests) where++import qualified Convex.Schema.Parser as Schema+import qualified Data.Map as Map+import Test.HUnit+import Text.Parsec (runParserT)++runSchemaTest :: String -> String -> Schema.ParsedFile -> Test+runSchemaTest testName input expected =+ testName ~: TestCase $ do+ result <- Schema.parseSchema input+ case result of+ Left err -> assertFailure ("Parser failed: " ++ show err)+ Right parsedFile -> parsedFile @?= expected++-- Test Case 1: A standard schema with mixed statement endings.+sampleSchemaStandard :: String+sampleSchemaStandard =+ unlines+ [ "import { defineSchema, defineTable, v } from \"convex/server\";",+ "",+ "export const roleEnum = v.union(v.literal(\"admin\"), v.literal(\"user\"));",+ "",+ "export const productEnum = v.union(v.literal(\"palaba\"), v.literal(\"audiorake\"))",+ "",+ "export default defineSchema({",+ " users: defineTable({",+ " name: v.string(),",+ " role: roleEnum",+ " }),",+ "});"+ ]++expectedSchemaStandard :: Schema.ParsedFile+expectedSchemaStandard =+ Schema.ParsedFile+ { Schema.parsedConstants =+ Map.fromList+ [ ("roleEnum", Schema.VUnion [Schema.VLiteral "admin", Schema.VLiteral "user"]),+ ("productEnum", Schema.VUnion [Schema.VLiteral "palaba", Schema.VLiteral "audiorake"])+ ],+ Schema.parsedSchema =+ Schema.Schema+ { Schema.getTables =+ [ Schema.Table+ { Schema.tableName = "users",+ Schema.tableFields =+ [ Schema.Field "name" Schema.VString,+ Schema.Field "role" (Schema.VReference "roleEnum")+ ],+ Schema.tableIndexes = []+ }+ ]+ }+ }++-- Test Case 2: Schema definition appears at the top of the file.+sampleSchemaAtTop :: String+sampleSchemaAtTop =+ unlines+ [ "import { defineSchema, defineTable, v } from \"convex/server\";",+ "",+ "export default defineSchema({",+ " projects: defineTable({",+ " name: v.string(),",+ " product: productEnum, // Reference to a const defined later",+ " }),",+ "});",+ "",+ "// This constant is defined after it is referenced.",+ "export const productEnum = v.union(v.literal(\"a\"), v.literal(\"b\"));"+ ]++expectedSchemaAtTop :: Schema.ParsedFile+expectedSchemaAtTop =+ Schema.ParsedFile+ { Schema.parsedConstants =+ Map.fromList+ [("productEnum", Schema.VUnion [Schema.VLiteral "a", Schema.VLiteral "b"])],+ Schema.parsedSchema =+ Schema.Schema+ { Schema.getTables =+ [ Schema.Table+ { Schema.tableName = "projects",+ Schema.tableFields =+ [ Schema.Field "name" Schema.VString,+ Schema.Field "product" (Schema.VReference "productEnum")+ ],+ Schema.tableIndexes = []+ }+ ]+ }+ }++-- Test Case 3: A table uses a reference to a const object for its definition.+sampleSchemaWithObjectRef :: String+sampleSchemaWithObjectRef =+ unlines+ [ "import { v, defineSchema, defineTable } from \"convex/server\";",+ "",+ "// A top-level constant defining the fields for a table.",+ "const authCodeValidator = {",+ " exchange_code: v.string(),",+ " tenant_id: v.id(\"tenants\"),",+ "};",+ "",+ "// A type alias, which should be parsed and stored.",+ "export type MyId = typeof v.id(\"tenants\");",+ "",+ "export default defineSchema({",+ " // This table uses the constant directly.",+ " auth_code_sessions: defineTable(authCodeValidator),",+ "});"+ ]++expectedSchemaWithObjectRef :: Schema.ParsedFile+expectedSchemaWithObjectRef =+ Schema.ParsedFile+ { Schema.parsedConstants =+ Map.fromList+ [ ( "authCodeValidator",+ Schema.VObject+ [ ("exchange_code", Schema.VString),+ ("tenant_id", Schema.VId "tenants")+ ]+ ),+ ("MyId", Schema.VId "tenants")+ ],+ Schema.parsedSchema =+ Schema.Schema+ { Schema.getTables =+ [ Schema.Table+ { Schema.tableName = "auth_code_sessions",+ -- The parser should expand the reference into the actual fields.+ Schema.tableFields =+ [ Schema.Field "exchange_code" Schema.VString,+ Schema.Field "tenant_id" (Schema.VId "tenants")+ ],+ Schema.tableIndexes = []+ }+ ]+ }+ }++sampleComplexNoSemicolons :: String+sampleComplexNoSemicolons =+ unlines+ [ "import { v, defineSchema, defineTable } from \"convex/server\"",+ "",+ "export const statusEnum = v.union(v.literal(\"pending\"), v.literal(\"complete\"))",+ "",+ "export default defineSchema({",+ " tasks: defineTable({",+ " title: v.string(),",+ " status: statusEnum,",+ " }).index(\"by_status\", [\"status\"])",+ " .index(\"by_title\", [\"title\"]),",+ "",+ " // This table uses the constant directly.",+ " sessions: defineTable(sessionValidator)",+ "})",+ "",+ "// This constant is defined after it is referenced.",+ "const sessionValidator = {",+ " userId: v.id(\"users\"),",+ " token: v.string()",+ "}",+ "",+ "export type ComplexType = typeof sessionValidator"+ ]++expectedComplexNoSemicolons :: Schema.ParsedFile+expectedComplexNoSemicolons =+ Schema.ParsedFile+ { Schema.parsedConstants =+ Map.fromList+ [ ("statusEnum", Schema.VUnion [Schema.VLiteral "pending", Schema.VLiteral "complete"]),+ ( "sessionValidator",+ Schema.VObject+ [ ("userId", Schema.VId "users"),+ ("token", Schema.VString)+ ]+ ),+ ("ComplexType", Schema.VReference "sessionValidator")+ ],+ Schema.parsedSchema =+ Schema.Schema+ { Schema.getTables =+ [ Schema.Table+ { Schema.tableName = "tasks",+ Schema.tableFields =+ [ Schema.Field "title" Schema.VString,+ Schema.Field "status" (Schema.VReference "statusEnum")+ ],+ Schema.tableIndexes = [Schema.Index "by_status" ["status"], Schema.Index "by_title" ["title"]]+ },+ Schema.Table+ { Schema.tableName = "sessions",+ Schema.tableFields =+ [ Schema.Field "userId" (Schema.VId "users"),+ Schema.Field "token" Schema.VString+ ],+ Schema.tableIndexes = []+ }+ ]+ }+ }++sampleOptionalArrayOfObjects :: String+sampleOptionalArrayOfObjects =+ unlines+ [ "import { v, defineSchema, defineTable } from \"convex/server\"",+ "export const accessLevelEnum = v.union(v.literal(\"read\"), v.literal(\"write\"))",+ "export default defineSchema({",+ " projects: defineTable({",+ " shared_users: v.optional(",+ " v.array(",+ " v.object({",+ " user_id: v.id(\"users\"),",+ " access: accessLevelEnum",+ " })",+ " )",+ " )",+ " })",+ "})"+ ]++expectedOptionalArrayOfObjects :: Schema.ParsedFile+expectedOptionalArrayOfObjects =+ Schema.ParsedFile+ { Schema.parsedConstants =+ Map.fromList+ [("accessLevelEnum", Schema.VUnion [Schema.VLiteral "read", Schema.VLiteral "write"])],+ Schema.parsedSchema =+ Schema.Schema+ { Schema.getTables =+ [ Schema.Table+ { Schema.tableName = "projects",+ Schema.tableFields =+ [ Schema.Field+ "shared_users"+ ( Schema.VOptional+ ( Schema.VArray+ ( Schema.VObject+ [ ("user_id", Schema.VId "users"),+ ("access", Schema.VReference "accessLevelEnum")+ ]+ )+ )+ )+ ],+ Schema.tableIndexes = []+ }+ ]+ }+ }++sampleArrayOfPrimitives :: String+sampleArrayOfPrimitives =+ unlines+ [ "import { v, defineSchema, defineTable } from \"convex/server\"",+ "export default defineSchema({",+ " analytics: defineTable({",+ " summary: v.array(v.number()),",+ " length: v.number(),",+ " })",+ "})"+ ]++expectedArrayOfPrimitives :: Schema.ParsedFile+expectedArrayOfPrimitives =+ Schema.ParsedFile+ { Schema.parsedConstants = Map.empty,+ Schema.parsedSchema =+ Schema.Schema+ { Schema.getTables =+ [ Schema.Table+ { Schema.tableName = "analytics",+ Schema.tableFields =+ [ Schema.Field "summary" (Schema.VArray Schema.VNumber),+ Schema.Field "length" Schema.VNumber+ ],+ Schema.tableIndexes = []+ }+ ]+ }+ }++-- Main test suite combining all cases.+tests :: Test+tests =+ "Schema Parser"+ ~: test+ [ runSchemaTest "parses standard schema with mixed endings" sampleSchemaStandard expectedSchemaStandard,+ runSchemaTest "parses schema defined before its constants" sampleSchemaAtTop expectedSchemaAtTop,+ runSchemaTest "parses table defined with an object reference" sampleSchemaWithObjectRef expectedSchemaWithObjectRef,+ runSchemaTest "parses complex schema without semicolons" sampleComplexNoSemicolons expectedComplexNoSemicolons,+ runSchemaTest "parses optional array of objects" sampleOptionalArrayOfObjects expectedOptionalArrayOfObjects,+ runSchemaTest "parses array of primitives" sampleArrayOfPrimitives expectedArrayOfPrimitives+ ]