Blog

  • vuls-to-redmine

    vuls-to-redmine

    vuls-log-converter を導入

    vulslogconverter https://github.com/usiusi360/vuls-log-converter でcsv に変換する

    json-to-diff を導入

    $ git clone https://github.com/nakacya/vuls-to-redmine
    

    json-to-diff.confを随時修正

    [API]
    oldpath=/tmp/old
    oldfiles=csvdata.csv
    newpath=/tmp/new
    newfiles=csvdata.csv
    path=/tmp/new
    files=output.csv
    

    vuls-log-converter を実行

    $ cd /tmp/old
    $ vulslogconverter -i /tmp/old -o /tmp/old/csvdata.csv -t csv
    $ cd /tmp/new
    $ vulslogconverter -i /tmp/new -o /tmp/new/csvdata.csv -t csv
    

    json-to-diff.pl を実行

    $ ./json-to-diff.pl -c json-to-diff.conf
    

    json-to-diff.pl は「全件マッチ」を行っている為非常に時間が掛かります。

    #1CPUだと10000レコードで20分位掛かります

    vuls-to-redmine を導入

    $ git clone https://github.com/nakacya/vuls-to-redmine
    

    perl module の追加

    Config::Tiny
    Text::CSV_XS
    LWP::UserAgent
    Crypt::SSLeay
    JSON
    Time::HiRes
    

    の perl module を追加

    param.conf を随時修正

    $ vi param.conf
    [API]
    key=redmine_api_key               #Redmine のアクセスキー
    project_id=1                      #Redmine の project_id
    tracker_id=1                      #Redmine の tracker_id
    assigned_to_id=5                  #Redmine の assigned_to_id
    status_id=1                       #Redmine の status_id
    closed_status_id=5                #Redmine への Close Ticket status_id
    path=csv_path                     #json-to-diff.plの出力先path
    files=output.csv                  #json-to-diff.pl の出力ファイル
    server=http://your_redmine_URL/   #Redmine の URL
    cvss=1                            #Redmine の cvssフィールドID
    method=2                          #Redmine の methodフィールドID
    notfix=3                          #Redmine の notfixフィールドID
    ssl_fail=0                        #Redmine サイトのSSL証明書の有効性チェック(0=無視)
    

    Redmine へカスタムフィールドを追加

    CVSS=小数
    DetectionMethod=リスト(OvalMatch/ChangelogExactMatch/CpeNameMatch)
    NotFixedYet=リスト(true/false/Unknown)
    を作成する
    

    json-to-diff.pl で差分取得した csv を vuls-to-redmine.pl にて実行

    $ ./vuls-to-redmine.pl -c param.conf
    

    使い方

    1:初回時

    初回だけは手動で以下のようなスクリプトを実行

    #!/bin/sh
    VULS_HOME="/opt/vuls"
    VULS_LOG="${VULS_HOME}/results"
    cd /tmp
    rm -rf /tmp/old /tmp/new
    mkdir /tmp/old /tmp/new
    touch /tmp/old/output.csv
    ls $VULS_LOG/current/*.json | grep  -v "_diff.json" | xargs -I{} cp {} /tmp/new
    /usr/bin/vulslogconv -i /tmp/new -o /tmp/new/csvdata.csv -t csv
    /opt/vuls/json-to-diff.pl -c /opt/vuls/json-to-diff_api.conf
    /opt/vuls/vuls-to-redmine.pl -c /opt/vuls/vuls-to-redmine_api.conf
    

    自分の意図とするredmineへの登録が正常に行われることを確認したら2:へ

    2:二回目以降

    vuls scan及びreport を実行の後に以下のようなスクリプトを実行

    #!/bin/sh
    VULS_HOME="/opt/vuls"
    VULS_LOG="${VULS_HOME}/results"
    cd /tmp
    rm -rf /tmp/old /tmp/new
    mkdir /tmp/old /tmp/new
    result_old = `find ${VULS_LOG} -maxdepth 1 -type d |sort -nr | head -2 | tail -1`
    ls $result_old/*.json | grep  -v "_diff.json" | xargs -I{} cp {} /tmp/old
    ls $VULS_LOG/current/*.json | grep  -v "_diff.json" | xargs -I{} cp {} /tmp/new
    /usr/bin/vulslogconv -i /tmp/old -o /tmp/old/csvdata.csv -t csv 
    /usr/bin/vulslogconv -i /tmp/new -o /tmp/new/csvdata.csv -t csv
    /opt/vuls/json-to-diff.pl -c /opt/vuls/json-to-diff_api.conf
    /opt/vuls/vuls-to-redmine.pl -c /opt/vuls/vuls-to-redmine_api.conf
    rm -rf /tmp/old /tmp/new
    

    判明している問題点

    ・redmineのチケットを手動でクローズ

    ・yum update で脆弱性対応

    ・json-to-diff.pl 及び vuls-to-redmine.pl を実行

    ・[[CLOSE!!]]の終了チケットが追加される

    または

    ・yum update後のデータを用いて vuls-to-redmine.pl を 複数回実行

    ・[[CLOSE!]]の終了チケットが追加される

    これは「クローズされたチケットの内容を確認しないと言う仕様」に基づくものですのでご了承ください。

    クローズされたチケットも確認するようにしてみた(実験中

    これで「ホスト名・パッケージ名・バージョン」を件名にしてCVSS/CWEはnoteとして一纏めに出来るはず?

    Visit original content creator repository
    https://github.com/nakacya/vuls-to-redmine

  • l10n_co_tax_extension

    Visit original content creator repository
    https://github.com/odooloco/l10n_co_tax_extension

  • trip-planner

    TripPlanner 🌍 A Travel Itinerary Web Application

    TripPlanner helps you create detailed travel plans with interactive maps, activity scheduling, and budget tracking.

    Home Page


    🔗 Live Demo – Try it now!

    Table of Contents

    Introduction

    TripPlanner is a comprehensive travel planning application that combines interactive mapping, detailed activity scheduling, and budget management.

    I created TripPlanner as both a learning experience and a practical solution for my own travel planning needs. This project allowed me to dive deep into React development, work with external APIs, and implement user authentication while building something I would actually use.

    TripPlanner is designed for desktop use only. For the best experience, please access the application on a desktop browser.

    TripPlanner in a nutshell:

    • 📍 Interactive map integration with HERE Maps API
    • 📅 Day-by-day activity planning
    • 💰 Budget tracking for each activity
    • 🔄 Real-time updates and synchronization
    • 🔐 Secure user authentication

    Key Features

    1. Interactive Trip Planning

    • Destination search with HERE Maps API autocomplete
    • Interactive map with location markers
    • Day-by-day itinerary planning

    2. Activity Management

    • Add, edit, and remove activities
    • Set start and end times
    • Add activity descriptions
    • Track costs per activity
    • Automatic budget calculation

    3. Map Integration

    • Interactive HERE Maps integration
    • Location search and suggestions
    • Visual representation of planned activities
    • Custom markers for each activity

    4. User Management

    • Secure authentication system using Firebase
    • Personal dashboard
    • Save and manage multiple trips
    • Simple profile customization

    5. Data Export & Sharing

    • Export itineraries to PDF
    • Simple sharing of trip content with others

    6. State Management

    • React Context API for global state
    • Custom reducers for predictable state updates
    • Real-time synchronization with Firestore
    • Local storage for guest users

    Note: The contact email shown in the footer of my application is not real and is included for demonstration purposes only.

    Tech Stack

    Frontend

    • React 18
    • Context API for state management
    • React Router for navigation
    • HTML5 & CSS3
    • JavaScript ES6+

    Backend & Services

    • Firebase Authentication
    • Firebase Firestore
    • HERE Maps API
    • Vite build tool
    • Vercel for deployment

    Additional Tools

    • Axios for API requests
    • html2pdf for PDF generation
    • Custom utility functions
    • Environment variable management

    Project Structure

    src/
    ├── components/
    │ ├── activity/ # Activity-related components
    │ ├── auth/ # Authentication components
    │ └── layout/ # Layout components
    ├── contexts/ # React Context providers
    ├── pages/ # Main application pages
    ├── reducers/ # State management reducers
    ├── styles/ # CSS stylesheets
    ├── utils/ # Utility functions
    └── config/ # Configuration files

    Acknowledgments

    A big thanks to Wanderlog for inspiring the design and functionality of this application. Their outstanding platform provided valuable insights that shaped my approach to building my TripPlanner.

    Preview

    Planning Page

    Planning Page

    Itinerary Page

    Itinerary Page

    Saved Trips Page

    Saved Trips Page



    Made with ❤️ by Cátia Monteiro. Thanks for reading!

    Visit original content creator repository https://github.com/diecatiamonteiro/trip-planner
  • feathers-sequelize

    feathers-sequelize

    CI Download Status Discord

    Caution: When you’re using feathers v4 and want to upgrade to feathers v5, please make sure to read the migration guide.

    NOTE: This is the version for Feathers v5. For Feathers v4 use feathers-sequelize v6

    A Feathers database adapter for Sequelize, an ORM for Node.js. It supports PostgreSQL, MySQL, MariaDB, SQLite and MSSQL and features transaction support, relations, read replication and more.

    Very Important: Before using this adapter you have to be familiar with both, the Feathers Basics and general use of Sequelize. For associations and relations see the associations section. This adapter may not cover all use cases but they can still be implemented using Sequelize models directly in a Custom Feathers service.

    npm install --save feathers-sequelize@pre

    And one of the following:

    npm install --save pg pg-hstore
    npm install --save mysql2 // For both mysql and mariadb dialects
    npm install --save sqlite3
    npm install --save tedious // MSSQL

    Important: feathers-sequelize implements the Feathers Common database adapter API and querying syntax. For more information about models and general Sequelize usage, follow up in the Sequelize documentation.

    API

    new SequelizeService(options)

    Returns a new service instance initialized with the given options.

    const Model = require('./models/mymodel');
    const { SequelizeService } = require('feathers-sequelize');
    
    app.use('/messages', new SequelizeService({ Model }));
    app.use('/messages', new SequelizeService({ Model, id, events, paginate }));

    Options:

    • Model (required) – The Sequelize model definition
    • id (optional, default: primary key of the model) – The name of the id field property. Will use the first property with primaryKey: true by default.
    • raw (optional, default: true) – Runs queries faster by returning plain objects instead of Sequelize models.
    • Sequelize (optional, default: Model.sequelize.Sequelize) – The Sequelize instance
    • events (optional) – A list of custom service events sent by this service
    • paginate (optional) – A pagination object containing a default and max page size
    • multi (optional) – Allow create with arrays and update and remove with id null to change multiple items. Can be true for all methods or an array of allowed methods (e.g. [ 'remove', 'create' ])
    • operatorMap (optional) – A mapping from query syntax property names to to Sequelize secure operators
    • operators (optional) – An array of additional query operators to allow (e..g [ '$regex', '$geoNear' ]). Default is the supported operators
    • filters (optional) – An object of additional query parameters to allow (e..g { '$post.id$': true }).`

    params.sequelize

    When making a service method call, params can contain an sequelize property which allows to pass additional Sequelize options. This can e.g. be used to retrieve associations. Normally this wil be set in a before hook:

    app.service('messages').hooks({
      before: {
        find(context) {
          // Get the Sequelize instance. In the generated application via:
          const sequelize = context.app.get('sequelizeClient');
          const { User } = sequelize.models;
    
          context.params.sequelize = {
            include: [ User ]
          }
    
          return context;
        }
      }
    });

    Other options that params.sequelize allows you to pass can be found in Sequelize querying docs. Beware that when setting a top-level where property (usually for querying based on a column on an associated model), the where in params.sequelize will overwrite your query.

    This library offers some additional functionality when using sequelize.returning in services that support multi. The multi option allows you to create, patch, and remove multiple records at once. When using sequelize.returning with multi, the sequelize.returning is used to indicate if the method should return any results. This is helpful when updating large numbers of records and you do not need the API (or events) to be bogged down with results.

    operatorMap

    Sequelize deprecated string based operators a while ago for security reasons. Starting at version 4.0.0 feathers-sequelize converts queries securely, so you can still use string based operators listed below. If you want to support additional Sequelize operators, the operatorMap service option can contain a mapping from query parameter name to Sequelize operator. By default supported are:

    '$eq',
    '$ne',
    '$gte',
    '$gt',
    '$lte',
    '$lt',
    '$in',
    '$nin',
    '$like',
    '$notLike',
    '$iLike',
    '$notILike',
    '$or',
    '$and'
    
    // Find all users with name similar to Dav
    app.service('users').find({
      query: {
        name: {
          $like: 'Dav%'
        }
      }
    });
    GET /users?name[$like]=Dav%
    

    Modifying the Model

    Sequelize allows you to call methods like Model.scope(), Model.schema(), and others. To use these methods, extend the class to overwrite the getModel method.

    const { SequelizeService } = require('feathers-sequelize');
    
    class Service extends SequelizeService {
      getModel(params) {
        let Model = this.options.Model;
        if (params?.sequelize?.scope) {
          Model = Model.scope(params.sequelize.scope);
        }
        if (params?.sequelize?.schema) {
          Model = Model.schema(params.sequelize.schema);
        }
        return Model;
      }
    }

    Caveats

    Sequelize raw queries

    By default, all feathers-sequelize operations will return raw data (using raw: true when querying the database). This results in faster execution and allows feathers-sequelize to interoperate with feathers-common hooks and other 3rd party integrations. However, this will bypass some of the “goodness” you get when using Sequelize as an ORM:

    • custom getters/setters will be bypassed
    • model-level validations are bypassed
    • associated data loads a bit differently
    • …and several other issues that one might not expect

    Don’t worry! The solution is easy. Please read the guides about working with model instances. You can also pass { raw: true/false} in params.sequelize to change the behavior per service call.

    Working with MSSQL

    When using MSSQL as the database, a default sort order always has to be applied, otherwise the adapter will throw an Invalid usage of the option NEXT in the FETCH statement. error. This can be done in your model with:

    model.beforeFind(model => model.order.push(['id', 'ASC']))

    Or in a hook like this:

    export default function (options = {}) {
      return async context => {
        const { query = {} } = context.params;
        // Sort by id field ascending (or any other property you want)
        // See https://docs.feathersjs.com/api/databases/querying.html#sort
        const $sort = { id: 1 };
    
        context.params.query = {
          $sort: {
    
          },
          ...query
        }
    
        return context;
      }
    }

    Primary keys

    All tables used by a feathers-sequelize service require a primary key. Although it is common practice for many-to-many tables to not have a primary key, this service will break if the table does not have a primary key. This is because most service methods require an ID and because of how feathers maps services to URLs.

    Example

    Here is an example of a Feathers server with a messages SQLite Sequelize Model:

    $ npm install @feathersjs/feathers @feathersjs/errors @feathersjs/express @feathersjs/socketio sequelize feathers-sequelize sqlite3
    

    In app.js:

    import path from 'path';
    import { feathers } from '@feathersjs/feathers';
    import express from '@feathersjs/express';
    import socketio from '@feathersjs/socketio';
    
    import Sequelize from 'sequelize';
    import SequelizeService from 'feathers-sequelize';
    
    const sequelize = new Sequelize('sequelize', '', '', {
      dialect: 'sqlite',
      storage: path.join(__dirname, 'db.sqlite'),
      logging: false
    });
    
    const Message = sequelize.define('message', {
      text: {
        type: Sequelize.STRING,
        allowNull: false
      }
    }, {
      freezeTableName: true
    });
    
    // Create an Express compatible Feathers application instance.
    const app = express(feathers());
    
    // Turn on JSON parser for REST services
    app.use(express.json());
    // Turn on URL-encoded parser for REST services
    app.use(express.urlencoded({ extended: true }));
    // Enable REST services
    app.configure(express.rest());
    // Enable Socket.io services
    app.configure(socketio());
    // Create an in-memory Feathers service with a default page size of 2 items
    // and a maximum size of 4
    app.use('/messages', new SequelizeService({
      Model: Message,
      paginate: {
        default: 2,
        max: 4
      }
    }));
    app.use(express.errorHandler());
    
    Message.sync({ force: true }).then(() => {
      // Create a dummy Message
      app.service('messages').create({
        text: 'Message created on server'
      }).then(message => console.log('Created message', message));
    });
    
    // Start the server
    const port = 3030;
    
    app.listen(port, () => {
      console.log(`Feathers server listening on port ${port}`);
    });

    Run the example with node app and go to localhost:3030/messages.

    Associations

    Embrace the ORM

    The documentation on Sequelize associations and relations is essential to implementing associations with this adapter and one of the steepest parts of the Sequelize learning curve. If you have never used an ORM, let it do a lot of the heavy lifting for you!

    Setting params.sequelize.include

    Once you understand how the include option works with Sequelize, you will want to set that option from a before hook in Feathers. Feathers will pass the value of context.params.sequelize as the options parameter for all Sequelize method calls. This is what your hook might look like:

    // GET /my-service?name=John&include=1
    function (context) {
      const { include, ...query } = context.params.query;
    
       if (include) {
          const AssociatedModel = context.app.services.fooservice.Model;
          context.params.sequelize = {
            include: [{ model: AssociatedModel }]
          };
          // Update the query to not include `include`
          context.params.query = query;
       }
    
       return context;
    }

    Underneath the hood, feathers will call your models find method sort of like this:

    // YourModel is a sequelize model
    const options = Object.assign({ where: { name: 'John' }}, context.params.sequelize);
    YourModel.findAndCount(options);

    For more information, follow up up in the Sequelize documentation for associations and this issue.

    Querying

    Additionally to the common querying mechanism this adapter also supports all Sequelize query operators.

    Querying a nested column

    To query based on a column in an associated model, you can use Sequelize’s nested column syntax in a query. The nested column syntax is considered a filter by Feathers, and so each such usage has to be whitelisted.

    Example:

    // Find a user with post.id == 120
    app.service('users').find({
      query: {
        '$post.id$': 120,
        include: {
          model: posts
        }
      }
    });

    For this case to work, you’ll need to add ‘$post.id$’ to the service options’ ‘filters’ property.

    Working with Sequelize Model instances

    It is highly recommended to use raw queries, which is the default. However, there are times when you will want to take advantage of Sequelize Instance methods. There are two ways to tell feathers to return Sequelize instances:

    1. Set { raw: false } in a “before” hook:

      const rawFalse = () => (context) => {
        if (!context.params.sequelize) context.params.sequelize = {};
        Object.assign(context.params.sequelize, { raw: false });
        return context;
      }
      
      export default {
        after: {
          // ...
          find: [rawFalse()]
          // ...
        },
        // ...
      };
    2. Use the hydrate hook in the “after” phase:

      import { hydrate } from 'feathers-sequelize';
      
      export default {
        after: {
          // ...
          find: [hydrate()]
          // ...
        },
        // ...
      };
      
      // Or, if you need to include associated models, you can do the following:
      const includeAssociated = () => (context) => hydrate({
        include: [{ model: context.app.services.fooservice.Model }]
      });
      
      export default {
        after: {
          // ...
          find: [includeAssociated()]
          // ...
        },
        // ...
      };

    For a more complete example see this gist.

    Important: When working with Sequelize Instances, most of the feathers-hooks-common will no longer work. If you need to use a common hook or other 3rd party hooks, you should use the “dehydrate” hook to convert data back to a plain object:

    import { dehydrate, hydrate } from 'feathers-sequelize';
    import { populate } = from 'feathers-hooks-common';
    
    export default {
      after: {
        // ...
        find: [hydrate(), doSomethingCustom(), dehydrate(), populate()]
        // ...
      },
      // ...
    };

    Validation

    Sequelize by default gives you the ability to add validations at the model level. Using an error handler like the one that comes with Feathers your validation errors will be formatted nicely right out of the box!

    Errors

    Errors do not contain Sequelize specific information. The original Sequelize error can be retrieved on the server via:

    import { ERROR } = from 'feathers-sequelize';
    
    try {
      await sequelizeService.doSomething();
    } catch(error) {
      // error is a FeathersError
      // Safely retrieve the Sequelize error
      const sequelizeError = error[ERROR];
    }

    Testing sequelize queries in isolation

    If you wish to use some of the more advanced features of sequelize, you should first test your queries in isolation (without feathers). Once your query is working, you can integrate it into your feathers app.

    1. Build a test file

    Create a temporary file in your project root like this:

    // test.js
    import app from from './src/app';
    // run setup to initialize relations
    app.setup();
    
    const seqClient = app.get('sequelizeClient');
    const SomeModel = seqClient.models['some-model'];
    const log = console.log.bind(console);
    
    SomeModel.findAll({
      /*
      * Build your custom query here. We will use this object later.
      */
    }).then(log).catch(log);

    And then run this file like this:

    node test.js
    

    Continue updating the file and running it until you are satisfied with the results.

    2. Integrate the query using a “before” hook

    Once your have your custom query working to your satisfaction, you will want to integrate it into your feathers app. Take the guts of the findAll operation above and create a “before” hook:

    function buildCustomQuery(context) {
    	context.params.sequelize = {
           /*
            * This is the same object you passed to "findAll" above.
            * This object is *shallow merged* onto the underlying query object
            * generated by feathers-sequelize (it is *not* a deep merge!).
            * The underlying data will already contain the following:
            *   - "where" condition based on query paramters
            *   - "limit" and "offset" based on pagination settings
            *   - "order" based $sort query parameter
            * You can override any/all of the underlying data by setting it here.
            * This gives you full control over the query object passed to sequelize!
            */
    	};
    }
    
    someService.hooks({
    	before: {
    		find: [buildCustomQuery]
    	}
    });

    Migrations

    Migrations with feathers and sequelize are quite simple. This guide will walk you through creating the recommended file structure, but you are free to rearrange things as you see fit. The following assumes you have a migrations folder in the root of your app.

    Initial Setup: one-time tasks

    npm install sequelize-cli --save -g
    
    • Create a .sequelizerc file in your project root with the following content:
    const path = require('path');
    
    module.exports = {
      'config': path.resolve('migrations/config.js'),
      'migrations-path': path.resolve('migrations/scripts'),
      'seeders-path': path.resolve('migrations/seeders'),
      'models-path': path.resolve('migrations/models.js')
    };
    • Create the migrations config in migrations/config.js:
    const app = require('../src/app');
    const env = process.env.NODE_ENV || 'development';
    const dialect = 'postgres'; // Or your dialect name
    
    module.exports = {
      [env]: {
        dialect,
        url: app.get(dialect),
        migrationStorageTableName: '_migrations'
      }
    };
    • Define your models config in migrations/models.js:
    const Sequelize = require('sequelize');
    const app = require('../src/app');
    const sequelize = app.get('sequelizeClient');
    const models = sequelize.models;
    
    // The export object must be a dictionary of model names -> models
    // It must also include sequelize (instance) and Sequelize (constructor) properties
    module.exports = Object.assign({
      Sequelize,
      sequelize
    }, models);

    Migrations workflow

    The migration commands will load your application and it is therefore required that you define the same environment variables as when running your application. For example, many applications will define the database connection string in the startup command:

    DATABASE_URL=postgres://user:pass@host:port/dbname npm start
    

    All of the following commands assume that you have defined the same environment variables used by your application.

    ProTip: To save typing, you can export environment variables for your current bash/terminal session:

    export DATABASE_URL=postgres://user:pass@host:port/db
    

    Create a new migration

    To create a new migration file, run the following command and provide a meaningful name:

    sequelize migration:create --name="meaningful-name"
    

    This will create a new file in the migrations/scripts folder. All migration file names will be prefixed with a sortable data/time string: 20160421135254-meaningful-name.js. This prefix is crucial for making sure your migrations are executed in the proper order.

    NOTE: The order of your migrations is determined by the alphabetical order of the migration scripts in the file system. The file names generated by the CLI tools will always ensure that the most recent migration comes last.

    Add the up/down scripts:

    Open the newly created migration file and write the code to both apply and undo the migration. Please refer to the sequelize migration functions for available operations. Do not be lazy – write the down script too and test! Here is an example of converting a NOT NULL column accept null values:

    'use strict';
    
    module.exports = {
      up: function (queryInterface, Sequelize) {
        return queryInterface.changeColumn('tableName', 'columnName', {
          type: Sequelize.STRING,
          allowNull: true
        });
      },
    
      down: function (queryInterface, Sequelize) {
        return queryInterface.changeColumn('tableName', 'columnName', {
          type: Sequelize.STRING,
          allowNull: false
        });
      }
    };

    ProTip: As of this writing, if you use the changeColumn method you must always specify the type, even if the type is not changing.

    ProTip: Down scripts are typically easy to create and should be nearly identical to the up script except with inverted logic and inverse method calls.

    Keeping your app code in sync with migrations

    The application code should always be up to date with the migrations. This allows the app to be freshly installed with everything up-to-date without running the migration scripts. Your migrations should also never break a freshly installed app. This often times requires that you perform any necessary checks before executing a task. For example, if you update a model to include a new field, your migration should first check to make sure that new field does not exist:

    'use strict';
    
    module.exports = {
      up: function (queryInterface, Sequelize) {
        return queryInterface.describeTable('tableName').then(attributes => {
          if ( !attributes.columnName ) {
            return queryInterface.addColumn('tableName', 'columnName', {
              type: Sequelize.INTEGER,
              defaultValue: 0
            });
          }
        })
      },
    
      down: function (queryInterface, Sequelize) {
        return queryInterface.describeTable('tableName').then(attributes => {
          if ( attributes.columnName ) {
            return queryInterface.removeColumn('tableName', 'columnName');
          }
        });
      }
    };

    Apply a migration

    The CLI tools will always run your migrations in the correct order and will keep track of which migrations have been applied and which have not. This data is stored in the database under the _migrations table. To ensure you are up to date, simply run the following:

    sequelize db:migrate
    

    ProTip: You can add the migrations script to your application startup command to ensure that all migrations have run every time your app is started. Try updating your package.json scripts attribute and run npm start:

    scripts: {
        start: "sequelize db:migrate && node src/"
    }
    

    Undo the previous migration

    To undo the last migration, run the following command:

    sequelize db:migrate:undo
    

    Continue running the command to undo each migration one at a time – the migrations will be undone in the proper order.

    Note: – You shouldn’t really have to undo a migration unless you are the one developing a new migration and you want to test that it works. Applications rarely have to revert to a previous state, but when they do you will be glad you took the time to write and test your down scripts!

    Reverting your app to a previous state

    In the unfortunate case where you must revert your app to a previous state, it is important to take your time and plan your method of attack. Every application is different and there is no one-size-fits-all strategy for rewinding an application. However, most applications should be able to follow these steps (order is important):

    1. Stop your application (kill the process)
    2. Find the last stable version of your app
    3. Count the number of migrations which have been added since that version
    4. Undo your migrations one at a time until the db is in the correct state
    5. Revert your code back to the previous state
    6. Start your app

    License

    Copyright (c) 2024

    Licensed under the MIT license.

    whitelist

    The whitelist property is no longer, you should use filters instead. Checkout the migration guide below.

    Feathers v5 introduces a convention for options.operators and options.filters. The way feathers-sequelize worked in previous version is not compatible with these conventions. Please read https://dove.feathersjs.com/guides/migrating.html#custom-filters-operators.

    Migrate to Feathers v5 (dove)

    There are several breaking changes for feathers-sequelize in Feathers v5. This guide will help you to migrate your existing Feathers v4 application to Feathers v5.

    Named export

    The default export of feathers-sequelize has been removed. You now have to import the SequelizeService class directly:

    import { SequelizeService } from 'feathers-sequelize';
    
    app.use('/messages', new SequelizeService({ ... }));

    This follows conventions from feathers v5.

    operators / operatorMap

    Feathers v5 introduces a convention for options.operators and options.filters. The way feathers-sequelize worked in previous version is not compatible with these conventions. Please read https://dove.feathersjs.com/guides/migrating.html#custom-filters-operators first.

    The old options.operators object is renamed to options.operatorMap:

    import { SequelizeService } from 'feathers-sequelize';
    import { Op } from 'sequelize';
    
    app.use('/messages', new SequelizeService({
      Model,
      // operators is now operatorMap:
      operatorMap: {
        $between: Op.between
      }
    }));

    filters

    Feathers v5 introduces a convention for options.operators and options.filters. The way feathers-sequelize worked in previous version is not compatible with these conventions. Please read https://dove.feathersjs.com/guides/migrating.html#custom-filters-operators first.

    Feathers v5 introduces a new filters option. It is an object to verify filters. Here you need to add $dollar.notation$ operators, if you have some.

    import { SequelizeService } from 'feathers-sequelize';
    
    app.use('/messages', new SequelizeService({
      Model,
      filters: {
        '$and': true,
        '$person.name$': true
      }
    }));
    Visit original content creator repository https://github.com/feathersjs-ecosystem/feathers-sequelize
  • assemblyline-v4-service

    Discord Static Badge Static Badge GitHub Issues or Pull Requests by label License

    Assemblyline 4 – Service Base

    This repository provides the base service functionality for Assemblyline 4 services.

    Image variants and tags

    Tag Type Description Example Tag
    latest The most recent build (can be unstable). latest
    build_type The type of build used. dev is the latest unstable build. stable is the latest stable build. stable or dev
    series Complete build details, including version and build type: version.buildType. 4.5.stable, 4.5.1.dev3

    Creating a new Assemblyline service

    You can create a new Assemblyline service by using this template:

    apt install jq
    pip install git+https://github.com/CybercentreCanada/assemblyline-service-template.git
    cruft create https://github.com/CybercentreCanada/assemblyline-service-template.git

    Documentation

    For more information about service development for Assemblyline, follow this guide.


    Assemblyline 4 – Service Base

    Ce référentiel fournit les fonctionnalités de base des services Assemblyline 4.

    Créer un nouveau service Assemblyline

    Vous pouvez créer un nouveau service Assemblyline en utilisant ce template.

    apt install jq
    pip install git+https://github.com/CybercentreCanada/assemblyline-service-template.git
    cruft create https://github.com/CybercentreCanada/assemblyline-service-template.git

    Variantes et étiquettes d’image

    Type d’étiquette Description Exemple d’étiquette
    dernière La version la plus récente (peut être instable). latest
    build_type Le type de compilation utilisé. dev est la dernière version instable. stable est la dernière version stable. stable or dev
    séries Le détail de compilation utilisé, incluant la version et le type de compilation : version.buildType. 4.5.stable, 4.5.1.dev3

    Documentation

    Pour plus d’informations sur le développement des services pour Assemblyline, suivez ce guide.

    Visit original content creator repository https://github.com/CybercentreCanada/assemblyline-v4-service
  • Voicent

    Voicent Python Library

    Production/Stable License: AGPL-3 ursais/Voicent

    The Voicent Python Simple Interface class contains the following functions:

    callText
    callAudio
    callStatus
    callRemove
    callTillConfirm
    importCampaign
    runCampaign
    importAndRunCampaign
    checkStatus
    exportResult

    These functions are used to invoke telephone calls from your Python program. For example, callText is used to call a specified number and automatically play your text message using text-to-speech engine.

    In order for this class to work, you’ll need to have Voicent Gateway installed somewhere in your network. This class simply sends HTTP request for telephone calls to the gateway. Voicent has a free edition for the gateway. You can download it from http://www.voicent.com.

    More information can be found at: http://www.voicent.com/devnet/docs/pyapi.htm

    Table of contents

    Install from PyPi using pip, a package manager for Python.

    pip install voicent-python

    Don’t have pip installed? Try installing it, by running this from the command line:

    $ curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python

    Or, you can download the source code (ZIP) for voicent, and then run:

    python setup.py install

    You may need to run the above commands with sudo.

    Bugs are tracked on GitHub Issues. In case of trouble, please check there if your issue has already been reported. If you spotted it first, help us smashing it by providing a detailed and welcomed feedback.

    This module is maintained by Open Source Integrators.

    Open Source Integrators

    Open Source Integrators™ (OSI) provides customers a unique combination of open source business process consulting and implementations.

    Current maintainer:

    max3903

    You are welcome to contribute. Please create an issue on Github to discuss how.

    Visit original content creator repository https://github.com/ursais/Voicent
  • CloudCore

    Visit original content creator repository
    https://github.com/deeje/CloudCore

  • olric-consul-plugin

    olric-consul-plugin

    This package implements ServiceDiscovery interface of Olric and uses Consul at background. With this plugin, you don’t need
    to maintain a static list of alive peers in the cluster.

    Install

    Get the code:

    go get -u github.com/olric-data/olric-consul-plugin
    

    Usage

    Load as compiled plugin

    Build

    With a properly configured Go environment:

    go build -buildmode=plugin -o consul.so 
    

    If you want to strip debug symbols from the produced binary, add -ldflags="-s -w" to build command.

    If you prefer to deploy Olric in client-server scenario, add a serviceDiscovery block to your olricd.yaml. A sample:

    serviceDiscovery:
      provider: "consul"
      path: "YOUR_PLUGIN_PATH/consul.so"
      address: "http://127.0.0.1:8500"
      passingOnly: true
      replaceExistingChecks: false
      insecureSkipVerify: true
      payload: '
          {
              "Name": "olric-cluster",
              "ID": "olric-node-1",
              "Tags": [
                "primary",
                "v1"
              ],
              "Address": "localhost",
              "Port": MEMBERLIST_PORT,
              "EnableTagOverride": false,
              "check": {
                "name": "Olric node on MEMBERLIST_PORT",
                "tcp": "0.0.0.0:MEMBERLIST_PORT",
                "interval": "10s",
                "timeout": "1s"
              }
          }
    '

    In embedded member deployment scenario:

    // import config package
    "github.com/olric-data/olric/config"
    
    // Get a new Olric config for local environment
    c := config.New("local")
    
    // Set service discovery definition
    sd := make(map[string]interface{})
    sd["provider"] = "consul"
    sd["path"] = "YOUR_PLUGIN_PATH/consul.so"
    sd["address"] = "http://127.0.0.1:8500"
    sd["passingOnly"] = true
    sd["payload"] = `{
      "Name": "olric-cluster",
      "ID": "olric-node-1",
      "Tags": [
        "primary",
        "v1"
      ],
      "Address": "localhost",
      "Port": MEMBERLIST_PORT,
      "EnableTagOverride": false,
      "check": {
        "name": "Check Olric node on MEMBERLIST_PORT",
        "tcp": "0.0.0.0:MEMBERLIST_PORT",
        "interval": "1s",
        "timeout": "1s"
      }
    }`
    
    c.ServiceDiscovery = sd

    Import directly as a library

    You can load the plugin directly as a library:

    import (
        olricconsul "github.com/olric-data/olric-consul-plugin/lib"
    )
    
    //...
    sd := make(map[string]interface{})
    sd["plugin"] = &olricconsul.ConsulDiscovery{}
    //...

    Configuration

    This plugin has very few configuration parameters:

    Parameter Description
    provider Name of the service discovery daemon. It’s Consul. Just informal
    path Absolute path of the compiled plugin
    plugin Pointer to imported plugin
    address Network address of the service discovery daemon
    passingOnly If you set this true, only healthy nodes will be discovered
    payload Service record for Consul
    replaceExistingChecks Missing healthchecks from the request will be deleted from the agent. Using this parameter allows to idempotently register a service and its checks without having to manually deregister checks.
    insecureSkipVerify Controls whether a client verifies the server’s certificate chain and host name. If insecureSkipVerify is true, TLS accepts any certificate presented by the server and any host name in that certificate.

    Please note that you cannot set plugin and path simultaneously. Olric chooses path if you set both of them.

    Contributions

    Please don’t hesitate to fork the project and send a pull request or just e-mail me to ask questions and share ideas.

    License

    The Apache License, Version 2.0 – see LICENSE for more details.

    Visit original content creator repository
    https://github.com/buraksezer/olric-consul-plugin

  • notion-utilities

    notion-utilities

    Library to perform basic transformation on notion database. It is possible to
    apply function on a property or combinations of properties in order to edit properties.

    ⚠ Use at your own risk. If you want to be safe, please apply the functions on copies of the
    properties you want to edit (Or at least when testing the proper working of your code).

    Installation

    pip install git+https://github.com/thomashirtz/notion-utilities#egg=notion-utilities

    How to set up ?

    1. Creation of an integration

    Create an integration for the notion-utilities library. The integration needs to target the workplace containing the database that will be modified.

    Option needed:

    • Read content
    • Update content
    • Insert content

    Copy the Internal Integration Token, it will be the token argument of the functions.

    2. Share the database with the Integration

    Go to your database in notion => Click on Share => Invite => Select the integration that you just created.

    Copy the link of the database (simply the URL on a browser, on the application => Click on ... => Copy Link).

    Extract the database_id from the URL : https://www.notion.so/<workspace_name>/<database_id>?v=<view_id>

    3. Run your scripts

    Create the script that you want to run and set the token and the database_id with the one got from the step 1 and the step 2, respectively.

    Examples

    Add a suffix, a prefix or apply a transformation

    from notion_utilities import apply_to_database
    from notion_utilities.properties import RichText
    
    
    def add_suffix_and_prefix(value):
      return 'suffix_' + value + '_prefix'
    
    
    token = ''
    database_id = ''
    
    if __name__ == '__main__':
      apply_to_database(
          token=token,
          database_id=database_id,
          source=RichText('Input'),
          target=RichText('Output'),
          function=add_suffix_and_prefix,
      )
    Possibility to use several properties and assign to several properties

    from notion_utilities import apply_to_database
    from notion_utilities.properties import RichText
    
    
    def transform(input_1, input_2):  
        # some useful transformation
        return output
    
    
    token = ''
    database_id = ''
    
    if __name__ == '__main__':
        apply_to_database(
            token=token,
            database_id=database_id,
            source=[RichText('Input 1'), RichText('Input 2')],
            target=RichText('Output'),
            function=transform,
        )

    from notion_utilities import apply_to_database
    from notion_utilities.properties import RichText
    
    
    def transform(input):  
        # some useful transformation
        return output_1, output_2
    
    
    token = ''
    database_id = ''
    
    if __name__ == '__main__':
        apply_to_database(
            token=token,
            database_id=database_id,
            source=RichText('Input'),
            target=[RichText('Output 1'), RichText('Output 2')],
            function=transform,
        )
    Property transformation (e.g. transform chinese characters to pinyin)

    import pinyin
    
    from notion_utilities import apply_to_database
    from notion_utilities.properties import RichText
    
    
    def get_pinyin(chinese):  
        return pinyin.get(chinese)
    
    
    token = ''
    database_id = ''
    
    if __name__ == '__main__':
        apply_to_database(
            token=token,
            database_id=database_id,
            source=RichText('Chinese'),
            target=RichText('Pinyin'),
            function=get_pinyin,
        )

    License

     Copyright 2022 Thomas Hirtz
    
     Licensed under the Apache License, Version 2.0 (the "License");
     you may not use this file except in compliance with the License.
     You may obtain a copy of the License at
    
         http://www.apache.org/licenses/LICENSE-2.0
    
     Unless required by applicable law or agreed to in writing, software
     distributed under the License is distributed on an "AS IS" BASIS,
     WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     See the License for the specific language governing permissions and
     limitations under the License.
    

    Visit original content creator repository
    https://github.com/thomashirtz/notion-utilities

  • grad-cam

    Grad-CAM: Gradient-weighted Class Activation Mapping

    Code for the paper

    Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization
    Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra
    https://arxiv.org/abs/1610.02391

    Demo: gradcam.cloudcv.org

    Overview

    Usage

    Download Caffe model(s) and prototxt for VGG-16/VGG-19/AlexNet using sh models/download_models.sh.

    Classification

    th classification.lua -input_image_path images/cat_dog.jpg -label 243 -gpuid 0
    th classification.lua -input_image_path images/cat_dog.jpg -label 283 -gpuid 0
    
    Options
    • proto_file: Path to the deploy.prototxt file for the CNN Caffe model. Default is models/VGG_ILSVRC_16_layers_deploy.prototxt
    • model_file: Path to the .caffemodel file for the CNN Caffe model. Default is models/VGG_ILSVRC_16_layers.caffemodel
    • input_image_path: Path to the input image. Default is images/cat_dog.jpg
    • input_sz: Input image size. Default is 224 (Change to 227 if using AlexNet)
    • layer_name: Layer to use for Grad-CAM. Default is relu5_3 (use relu5_4 for VGG-19 and relu5 for AlexNet)
    • label: Class label to generate grad-CAM for (-1 = use predicted class, 283 = Tiger cat, 243 = Boxer). Default is -1. These correspond to ILSVRC synset IDs
    • out_path: Path to save images in. Default is output/
    • gpuid: 0-indexed id of GPU to use. Default is -1 = CPU
    • backend: Backend to use with loadcaffe. Default is nn
    • save_as_heatmap: Whether to save heatmap or raw Grad-CAM. 1 = save heatmap, 0 = save raw Grad-CAM. Default is 1
    Examples

    ‘border collie’ (233)

    ‘tabby cat’ (282)

    ‘boxer’ (243)

    ‘tiger cat’ (283)

    Visual Question Answering

    Clone the VQA (http://arxiv.org/abs/1505.00468) sub-repository (git submodule init && git submodule update), and download and unzip the provided extracted features and pretrained model.

    th visual_question_answering.lua -input_image_path images/cat_dog.jpg -question 'What animal?' -answer 'dog' -gpuid 0
    th visual_question_answering.lua -input_image_path images/cat_dog.jpg -question 'What animal?' -answer 'cat' -gpuid 0
    
    
    Options
    • proto_file: Path to the deploy.prototxt file for the CNN Caffe model. Default is models/VGG_ILSVRC_19_layers_deploy.prototxt
    • model_file: Path to the .caffemodel file for the CNN Caffe model. Default is models/VGG_ILSVRC_19_layers.caffemodel
    • input_image_path: Path to the input image. Default is images/cat_dog.jpg
    • input_sz: Input image size. Default is 224 (Change to 227 if using AlexNet)
    • layer_name: Layer to use for Grad-CAM. Default is relu5_4 (use relu5_3 for VGG-16 and relu5 for AlexNet)
    • question: Input question. Default is What animal?
    • answer: Optional answer (For eg. “cat”) to generate Grad-CAM for (” = use predicted answer). Default is ”
    • out_path: Path to save images in. Default is output/
    • model_path: Path to VQA model checkpoint. Default is VQA_LSTM_CNN/lstm.t7
    • gpuid: 0-indexed id of GPU to use. Default is -1 = CPU
    • backend: Backend to use with loadcaffe. Default is cudnn
    • save_as_heatmap: Whether to save heatmap or raw Grad-CAM. 1 = save heatmap, 0 = save raw Grad-CAM. Default is 1
    Examples

    What animal? Dog

    What animal? Cat

    What color is the fire hydrant? Green

    What color is the fire hydrant? Yellow

    What color is the fire hydrant? Green and Yellow

    What color is the fire hydrant? Red and Yellow

    Image Captioning

    Clone the neuraltalk2 sub-repository. Running sh models/download_models.sh will download the pretrained model and place it in the neuraltalk2 folder.

    Change lines 2-4 of neuraltalk2/misc/LanguageModel.lua to the following:

    local utils = require 'neuraltalk2.misc.utils'
    local net_utils = require 'neuraltalk2.misc.net_utils'
    local LSTM = require 'neuraltalk2.misc.LSTM'
    
    th captioning.lua -input_image_path images/cat_dog.jpg -caption 'a dog and cat posing for a picture' -gpuid 0
    th captioning.lua -input_image_path images/cat_dog.jpg -caption '' -gpuid 0
    
    
    Options
    • input_image_path: Path to the input image. Default is images/cat_dog.jpg
    • input_sz: Input image size. Default is 224 (Change to 227 if using AlexNet)
    • layer: Layer to use for Grad-CAM. Default is 30 (relu5_3 for vgg16)
    • caption: Optional input caption. No input will use the generated caption as default
    • out_path: Path to save images in. Default is output/
    • model_path: Path to captioning model checkpoint. Default is neuraltalk2/model_id1-501-1448236541.t7
    • gpuid: 0-indexed id of GPU to use. Default is -1 = CPU
    • backend: Backend to use with loadcaffe. Default is cudnn
    • save_as_heatmap: Whether to save heatmap or raw Grad-CAM. 1 = save heatmap, 0 = save raw Grad-CAM. Default is 1
    Examples

    a dog and cat posing for a picture

    a bathroom with a toilet and a sink

    License

    BSD

    3rd-party

    Visit original content creator repository https://github.com/ramprs/grad-cam