Thread: Developing, Debugging and Testing your first strategy (upd)

Developing, Debugging and Testing Your First Strategy

This article guides you trough all stages of strategy development: writing code, debugging and back testing.

Prerequisites

Please download and install Indicator SDK. You can download it by this link: download

Please familiarize yourself with the indicators development first. Read and try at least the first article of the listed below.
Guppy's Multiple Moving Average (Development Example: Guppy's Multiple Moving Average)
Piercing Line pattern detection (Development Example: Piercing Line pattern detection)
Yesterday's Close Value (Development Example: Yesterday's Close Value)

Strategy to Develop

Let’s develop the simplest strategy which buys when the fast moving average crosses over the slow moving average and sells when the fast moving average crosses under the slow moving average. The user can choose the timeframe at which the strategy can be applied, the account to trade on, the lot size to trade, and setup stop and limit orders.

Our strategy can work on both NFA-regulated and non-NFA accounts. However, the stop and limits aren’t available for NFA-regulated accounts.

Important note about the timeframe: The user can choose either “tick” or “bar” source. In case the “bar” source is chosen, we take into account the recently closed candle only. Of course, the strategy can work in such way that the currently changing candle can be processed as well, but for many applications such approach provides too many “noisy” signals (we’ve discussed this approach in a separate article).

Step 1. Writing Source Code

The strategy source is a plain text file which contains the source code written in Lua. You can use any of your preferable text editors. The syntax highlighting is welcome, but not necessary. In this article I use the SDK’s editor, but you can easily do the same steps in any other programmer’s editor.

So, first, start the editor application. (StartMenu->Programs->IndicoreSDK->Lua Editor).

Let’s name the file and save it into the proper place first. Choose File->Save As command. The “Save As” form appears. By default, the editor opens and saves files from the indicators’ folder of SDK. Strategy must be saved into the strategies’ folder of the SDK, so go to the parent folder and then to the sub-folder named “strategies”. Then name the strategy. This name must be unique, so choose it carefully. Let’s name our strategy - “sample”.



Now we can create the “strategy skeleton”. There are a number of functions we must implement in the strategy.

The first function is Init(). This function “introduces” the strategy for Marketscope. It is called once, when Marketscope initially loads the strategy. This function must provide the strategy name, description and the set of parameters which must be prompted when the user runs the strategy.

The second function is Prepare(). This function is called every time when the user filled the parameters but before the first price update. Your strategy must validate all the parameters’ values and prepare all the information for further execution.

The third function is ExtUpdate(). This function is called every time another tick (for tick price sources) comes or another bar (for bar sources) is closed.

And finally, we must include the Lua’s “helper” functions which simplify the strategy development process.

Note: Yes, I know. You have already read the “strategy structure” article in the user guide and wonder where the Update() function is. Ok. This function is a part of “pure” strategies which listen only ticks and manage other timeframes by themselves. We are using a simpler approach in this example, based on the above mentioned helper functions. You can read about these helpers here in this article in the user guide.

Code:

Code: Select all
function Init()
    strategy:name("Sample Moving Average Strategy");
    strategy:description("Just a sample which buys on fast crosses over slow and sells when slow crosses over fast");
end

function Prepare()
end

function ExtUpdate(id, source, period)
end

Now, let’s prepare the list of parameters to be prompted when the user applies our strategy. The instrument will be prompted automatically, but there is a lot of information to ask:

We must know:
1) The number of periods to calculate fast and slow moving averages.
2) The kind of the price (bid or ask) and the time frame the user wants to apply our strategy on.
3) Whether the user wants to see text alert and/or sound when the trading condition is met.
4) Whether the user wants to let our strategy trade.
5) The account to trade on.
6) The default trade size.
7) Whether the user wants to use stop and limit orders for the risk management.

All these things to be prompted are called parameters. The parameters can be an integer or a real number, a string, a file name, a Boolean (yes/no) value and even the so-called “lists”. “A list” is when the user doesn’t enter the value manually but chooses one of the values we provided as an “alternatives”. Also, some parameters can have additional predefined behavior. For example, the parameter enables the user to choose one of the existing accounts or one of the supported time frames. To activate such predefined behavior, we use “flags”.

Each parameter is described by the identifier. We will use this identifier to get the parameter value. This must be formed using the same rules as the Lua identifier (can include English letters, numbers and underscore sing). Also, we must provide the name and the description to be shown to the user, and the default value. The numeric parameters can also have minimum and maximum values.

So, just add the parameter set to the Init() function. Please note that we split parameters into “groups”, so the user could navigate through them a bit easier.

Code:

    strategy.parameters:addGroup("Moving Average Parameters");
    strategy.parameters:addInteger("F", "Fast Periods", "", 5, 1, 200);
    strategy.parameters:addInteger("S", "Slow Periods", "", 20, 1, 200);

    strategy.parameters:addGroup("Price");
    strategy.parameters:addString("PT", "Price Type", "", "Bid");
    strategy.parameters:addStringAlternative("PT", "Bid", "", "Bid");
    strategy.parameters:addStringAlternative("PT", "Bid", "", "Ask");
    strategy.parameters:addString("TF", "Time Frame", "", "m1");
    strategy.parameters:setFlag("TF", core.FLAG_PERIODS);

    strategy.parameters:addGroup("Signals");
    strategy.parameters:addBoolean("ShowAlert", "Show Alert", "", true);
    strategy.parameters:addBoolean("PlaySound", "Play Sound", "", false);
    strategy.parameters:addFile("SoundFile", "Sound File", "", "");
    strategy.parameters:setFlag("SoundFile", core.FLAG_SOUND);


    strategy.parameters:addGroup("Trading");
    strategy.parameters:addBoolean("CanTrade", "Allow Trading", "", true);
    strategy.parameters:addString("Account", "Account to trade", "", "");
    strategy.parameters:setFlag("Account", core.FLAG_ACCOUNT);
    strategy.parameters:addInteger("LotSize", "Size of the trade in lots", "", 1, 1, 100);
    strategy.parameters:addInteger("Stop", "Distance in pips for the stop order", "Use 0 to do not use stops", 0, 1, 100);
    strategy.parameters:addInteger("Limit", "Distance in pips for the limit order", "Use 0 to do not use limits", 0, 1, 100);

So, we have introduced the strategy to Marketscope. Marketscope can show our strategy in the list and can show the parameters and prompt for their values when the user wants to run the strategy.

Now let’s start working on the Prepare() method. Let’s go step-by-step.

First, check whether the fast moving average is really faster than the slow.

Code:

    local F, S;
    F = instance.parameters.F;
    S = instance.parameters.S;
    assert(F < S, "The fast moving average must be faster than the slow");

Note: The assert function throws an error with the text specified in the second parameter when the condition specified in the first parameter fails.

Then check the sound alert parameters. The sound file must be specified in case the user requested to play the sound. Also, keep the sound file name for further usage in the global variable.

Code:

local SoundFile;

function Prepare()
    ...
    local ShowAlert;
    ShowAlert = instance.parameters.ShowAlert;
    if instance.parameters.PlaySound then
        SoundFile = instance.parameters.SoundFile;
    else
        SoundFile = nil;
    end
    assert(not(PlaySound) or (PlaySound and SoundFile ~= ""), "Sound file must be specified");

Then check whether we can trade and prepare all the trading related-data. Keep the trading data in the global variables as well. Just collect the data first.

Code:

local CanTrade;
local Amount;
local Account;
local Stop;
local Limit;
local CanClose;
local CanStop;
local OfferID;

function Prepare()
    ...
    if CanTrade then
        Account = instance.parameters.Account;
        Stop = math.floor(instance.parameters.Stop + 0.5);
        Limit = math.floor(instance.parameters.Limit + 0.5);
        local instrument = instance.bid:instrument();
    end

Now we have the instrument name (for example, EUR/USD). But for trading operations we must have OfferID – the identifier of the instrument inside the trading system. We can get this identifier in the offers trading table. Use the host table to get the offers trading table and then find the instrument name inside this table.

Code:

        OfferID = core.host:findTable("offers"):find("Instrument", instrument).OfferID;

The code line above gets the “offers” trading table, then searches inside this trading table for a row in which the “instrument” column is equal to our instrument and then takes “OfferID” column of that row. We don’t check for successful search here because if the user can choose the instrument, this instrument definitely exists in the offers trading table.

Then we have to check whether Stop and Limit orders can be used. Some accounts are under NFA Rule 2-43(b), so neither close nor stop and limit orders can be used on such accounts. We can use the host:execute(“getTradingProperty”) method to check almost all trading-related rules.

Let’s check whether stop, limit and close orders are permitted.

continued.

Code:

        CanStop = core.host:execute("getTradingProperty", "canCreateStopLimit", instrument, Account);
        CanClose = core.host:execute("getTradingProperty", "canCreateMarketClose", instrument, Account);

Then, we must convert the amount specified in lots into the absolute amount as the trading system requires for the orders. We use the trading properties again to convert the lot into the absolute amount for the particular instrument and account.

Code:

        Amount = instance.parameters.LotSize * core.host:execute("getTradingProperty", "baseUnitSize", instrument, Account);

Then we must provide the name of the instance of the strategy. Usually, the instance is named as the strategy identifier plus the set of parameters used to run the strategy. Just make the name and then set it up for the strategy instance and for the alerts. Alerts will show this name in front of the message.

Code:

    local name = profile:id() .. "(" .. instance.bid:name() .. "," .. instance.parameters.TF .. "," .. F .. "," .. S .. ")";
    instance:name(name);
    ExtSetupSignal(name, ShowAlert);

And, finally, we have to subscribe for tick updates to let our strategy be activated the first time.

Code:

ExtSubscribe(1, nil, "t1", true, "tick");

Please, pay attention, we do NOT subscribe for the chosen time frame right in the Prepare() function. This function is also called every time the user changes any parameter in the “Add Strategy” function, so, if we put our subscription code into the Prepare() function, it will subscribe/unsubscribe multiple times, every time the user changes the parameter value. The tick subscription is always available for the strategy, so it does not increase resource usage, unlike any bar subscription.

Ok. We have finished with the Prepare() function.

Let’s start with the strategy logic. All the logic must be done inside the ExtUpdate() function or the functions called from this function. This function is called every time a new tick appears (for the tick subscriptions) or the bar has been closed (so the strategy is called only once for each bar).

The first thing to do is to subscribe to the prices in the timeframe which were actually chosen by the user.

Add a global variable to keep the reference to the price history we are subscribed for. Also, add the global variables to keep the indicators we want to create. The tick subscription has identifier 1 (see ExtSubscribe function above), so subscribe for the prices if the strategy is called for the update of subscription 1 and we haven’t subscribed for the chosen time frame yet.

Code:

local price = nil;
local fastMA;
local slowMA;
local first;

function ExtUpdate(id, source, period)
    if id == 1 and price == nil then
        price = ExtSubscribe(2, nil, instance.parameters.TF, true, "close");
        fastMA = core.indicators:create("MVA", price, instance.parameters.F);
        slowMA = core.indicators:create("MVA", price, instance.parameters.S);
        first = math.max(fastMA.DATA:first(), slowMA.DATA:first()) + 1;
    end
end

The code above subscribes for the prices in the chosen time frame, requests only close prices for the bars (or just ticks in case the user has chosen ticks) and then creates two MVA indicators applied on these prices. Please note that we also keep the “first” value. The first value is the oldest bar for which the indicator can be calculated. For example, MVA(7) cannot be calculated for the first 6 bars, because it’s a sum of the last 7 bars (including the current) divided by 7. “+1” is because we plan to check “cross” condition which requires that the previous bar value is also available. So, for MVA(7) the first bar we can process is 8th bar.

Now we can provide our logic. But before writing the logic itself, let’s create a couple of functions to trade. One function is for opening a trade and another for closing all existing trades in the specified direction.

To open the trade, we must know direction (buy or sell), amount, account and offer. We already know everything except the direction. So, the only parameter of our function must be a flag indicating whether a sell or a buy position must be opened. To execute the trading command, we use terminal:execute() method. First of all, we must create and fill a valuemap with all known parameters (order type (“Open Market”), account, offer and amount:

Code:

function open(sell)
    
    valuemap = core.valuemap();
    valuemap.Command = "CreateOrder";
    valuemap.OrderType = "OM";
    valuemap.OfferID = OfferID;
    valuemap.AcctID = Account;
    valuemap.Quantity = Amount;
    ...
end

Then we must write the side – “S” for sell order and “B” for buy order and calculate the stop and limit levels against the current bid/ask prices using the offset of the stop and limit orders specified in pips in the parameters. Pay attention that we calculate stop and limit levels against different prices to avoid having the stop or limit inside the spread.

Code:

    local side, limit, stop;
    if sell then
        side = "S";
        stop = instance.ask[instance.ask:size() - 1] + Stop * instance.ask:pipSize();
        limit = instance.bid[instance.ask:size() - 1] - Limit * instance.bid:pipSize();
    else
        side = "B";
        stop = instance.bid[instance.ask:size() - 1] - Stop * instance.ask:pipSize();
        limit = instance.ask[instance.ask:size() - 1] + Limit * instance.bid:pipSize();
    end

    valuemap.BuySell = side;

    if Stop > 0 and CanStop then
        valuemap.RateStop = stop;
    end

    if Limit > 0 and CanStop then
        valuemap.RateLimit = limit;
    end

Now the order is ready to be sent. Just do it and check whether the order parameters have been filled successfully.

Code:

    local success, msg;
    success, msg = terminal:execute(200, valuemap);
    assert(success, msg);

Now collect all code above into the function.

Code:

function open(sell)
    local valuemap;
    valuemap = core.valuemap();
    valuemap.Command = "CreateOrder";
    valuemap.OrderType = "OM";
    valuemap.OfferID = OfferID;
    valuemap.AcctID = Account;
    valuemap.Quantity = Amount;
    
    local side, limit, stop;
    if sell then
        side = "S";
        stop = instance.ask[instance.ask:size() - 1] + Stop * instance.ask:pipSize();
        limit = instance.bid[instance.ask:size() - 1] - Limit * instance.bid:pipSize();
    else
        side = "B";
        stop = instance.bid[instance.ask:size() - 1] - Stop * instance.ask:pipSize();
        limit = instance.ask[instance.ask:size() - 1] + Limit * instance.bid:pipSize();
    end

    valuemap.BuySell = side;

    if Stop > 0 and CanStop then
        valuemap.RateStop = stop;
    end

    if Limit > 0 and CanStop then
        valuemap.RateLimit = limit;
    end
    
    local success, msg;
    success, msg = terminal:execute(200, valuemap);
    assert(success, msg);
end

Now we can create the function for closing the trades. We need to use “CM” (close market) order for each trade which is opened in the specified direction on the specified account and the specified instrument. To get a list of all open trades, we must enumerate the “trades” trading table.

Code:

function close(sell)
    local enum, side;

    if sell then
        side = "S";
    else
        side = "B";
    end

   enum = core.host:findTable("trades"):enumerator();
   while true do
        row = enum:next();
        if row == nil then
            break;
        end
        if row.AccountID == Account and row.OfferID == OfferID and row.BS == side then
            -- row contains an open trade on our account and instrument in the specified direction (buy or sell).
        end
   end  
end

Now add the close order for each trade which must be closed.

Code:

function close(sell)
    local enum, side, valuemap;

    if sell then
        side = "S";
    else
        side = "B";
    end

   enum = core.host:findTable("trades"):enumerator();
   while true do
        row = enum:next();
        if row == nil then
            break;
        end
        if row.AccountID == Account and row.OfferID == OfferID and row.BS == side then
                valuemap = core.valuemap();
                valuemap.Command = "CreateOrder";
                valuemap.OrderType = "CM";
                valuemap.OfferID = OfferID;
                valuemap.AcctID = Account;
                valuemap.Quantity = row.Lot;
                valuemap.TradeID = row.TradeID;
                if row.BS == "B" then
                    valuemap.BuySell = "S";
                else
                    valuemap.BuySell = "B";
                end
                local success, msg;
                success, msg = terminal:execute(200, valuemap);
                assert(success, msg);
        end
   end  
end

Now we can close all the existing trades in the specified direction. But what about NFA accounts, for which we just cannot use close orders? It’s simple. Just create an opposite open market orders! We can do it because the accounts with the close market orders disabled are never hedging accounts. So, the final version of the close function is:

Code:

function close(sell)
    local enum, row, valuemap, side, closed;
    local valuemap;

    if sell then
        side = "S";
    else
        side = "B";
    end
    closed = false;

    enum = core.host:findTable("trades"):enumerator();
    while true do
        row = enum:next();
        if row == nil then
            break;
        end
        if row.AccountID == Account and
           row.OfferID == OfferID and
           row.BS == side then
            if CanClose then
                -- non-NFA accounts
                valuemap = core.valuemap();
                valuemap.Command = "CreateOrder";
                valuemap.OrderType = "CM";
                valuemap.OfferID = OfferID;
                valuemap.AcctID = Account;
                valuemap.Quantity = row.Lot;
                valuemap.TradeID = row.TradeID;
                if row.BS == "B" then
                    valuemap.BuySell = "S";
                else
                    valuemap.BuySell = "B";
                end
                local success, msg;
                success, msg = terminal:execute(200, valuemap);
                assert(success, msg);
            else
                -- NFA accounts
                valuemap = core.valuemap();
                valuemap.OrderType = "OM";
                valuemap.Command = "CreateOrder";
                valuemap.OfferID = OfferID;
                valuemap.AcctID = Account;
                valuemap.Quantity = row.Lot;
                if row.BS == "B" then
                    valuemap.BuySell = "S";
                else
                    valuemap.BuySell = "B";
                end
                local success, msg;
                success, msg = terminal:execute(200, valuemap);
                assert(success, msg);
            end
            closed = true;
        end
    end
    return closed;
end

So, we have all required things. Now just add the code which is activated every time another candle is closed. As you remember, we subscribed for the history with identifier “2”. So, every time the candle is closed, we must update the indicators and then check whether these indicators cross.

So, let’s return to the ExtUpdate function and add the following code there.

Code:

    elseif id == 2 then
        fastMA:update(core.UpdateLast);
        slowMA:update(core.UpdateLast);
        if period >= first then
           if core.crossesOver(fastMA.DATA, slowMA.DATA, period) then
               ExtSignal(instance.ask, instance.ask:size() - 1, "BUY", SoundFile, nil);
           elseif core.crossesUnder(fastMA.DATA, slowMA.DATA, period) then
               ExtSignal(instance.bid, instance.bid:size() - 1, "SELL", SoundFile, nil);
           end
       end

Now the strategy shows the alert requested (text, sound or both) when indicators cross. We also must add the trading code in case the trading is allowed. For buy signal we must close all sell positions or open new buy in case there is no sell position. For the sell signal we must close all buy positions or open a sell position in case there is no buy position.

So, add a flag into the “close” function which indicates whether any of the positions are closed. Then add the following code inside “buy” if statement

Code:

               if CanTrade then
                   if not(close(true)) then
                       open(false);
                   end
               end

And, accordingly, add the following code inside “sell” if statement

Code:

               if CanTrade then
                   if not(close(false)) then
                       open(true);
                   end
               end

That’s all. Now see the whole code of our strategy. I also added a few comments to make the navigation trough the code easier.

Download the code:


[/CODE]

continued...
[CODE]

Code:

-- Intoduce the strategy to the host application (for example, Marketscope).
-- The function is called once when the host application initially loads the strategy.
function Init()
    -- User-friendly name and the description
    strategy:name("Sample Moving Average Strategy");
    strategy:description("Just a sample which buys on fast crosses over slow and sells when slow crosses over fast");

    -- Fast and slow moving average parameters
    strategy.parameters:addGroup("Moving Average Parameters");
    strategy.parameters:addInteger("F", "Fast Periods", "", 5, 1, 200);
    strategy.parameters:addInteger("S", "Slow Periods", "", 20, 1, 200);

    -- Price subscription parameters (bid or ask price, time frame)
    strategy.parameters:addGroup("Price");
    strategy.parameters:addString("PT", "Price Type", "", "Bid");
    strategy.parameters:addStringAlternative("PT", "Bid", "", "Bid");
    strategy.parameters:addStringAlternative("PT", "Ask", "", "Ask");
    strategy.parameters:addString("TF", "Time Frame", "", "m1");
    strategy.parameters:setFlag("TF", core.FLAG_PERIODS);

    -- Alert parameters
    strategy.parameters:addGroup("Alerts");
    strategy.parameters:addBoolean("ShowAlert", "Show Alert", "", true);
    strategy.parameters:addBoolean("PlaySound", "Play Sound", "", false);
    strategy.parameters:addFile("SoundFile", "Sound File", "", "");
    strategy.parameters:setFlag("SoundFile", core.FLAG_SOUND);

    -- Trading parameters
    strategy.parameters:addGroup("Trading");
    strategy.parameters:addBoolean("CanTrade", "Allow Trading", "", true);
    strategy.parameters:addString("Account", "Account to trade", "", "");
    strategy.parameters:setFlag("Account", core.FLAG_ACCOUNT);
    strategy.parameters:addInteger("LotSize", "Size of the trade in lots", "", 1, 1, 100);
    strategy.parameters:addInteger("Stop", "Distance in pips for the stop order", "Use 0 to do not use stops", 0, 0, 100);
    strategy.parameters:addInteger("Limit", "Distance in pips for the limit order", "Use 0 to do not use limits", 0, 0, 100);
end

-- The global variables
local price = nil;      -- the price history we subscribed for
local fastMA;           -- fast moving average indicator
local slowMA;           -- slow moving average indicator
local first;            -- the index of the oldest period where we can check whether moving averages has been crossed
local CanTrade;         -- flag indicating whether we can trade
local Amount;           -- the amount for the trade
local Account;          -- the account to trade on
local Stop;             -- the stop order level expressed in pips or 0 if no stop must be used
local Limit;            -- the limit order level expressed in pips or 0 if no stop must be used
local CanClose;         -- the flag indicating whether "close market" orders is allowed
local CanStop;
local OfferID;          -- the internal indentifier of the instrument (required for the orders)
local SoundFile;        -- the sound file name or nil if no sound must be played

-- Prepare all the data.
-- The function is called once when the strategy is about to be started.
function Prepare()
    -- check moving average parameters
    local F, S;
    F = instance.parameters.F;
    S = instance.parameters.S;
    assert(F < S, "The fast moving average must be faster than the slow");

    -- check alerts settings
    local ShowAlert;
    ShowAlert = instance.parameters.ShowAlert;
    if instance.parameters.PlaySound then
        SoundFile = instance.parameters.SoundFile;
    else
        SoundFile = nil;
    end
    assert(not(PlaySound) or (PlaySound and SoundFile ~= ""), "Sound file must be specified");

    -- check whether the strategy is allowed to trade
    CanTrade = instance.parameters.CanTrade;
    if CanTrade then
        -- Prepare the common information (account, stop, limit and OfferID (internal id of the instrument).
        Account = instance.parameters.Account;
        Stop = math.floor(instance.parameters.Stop + 0.5);
        Limit = math.floor(instance.parameters.Limit + 0.5);
        local instrument = instance.bid:instrument();
        OfferID = core.host:findTable("offers"):find("Instrument", instrument).OfferID;
        -- check whether stop and limit orders are allowed
        CanStop = core.host:execute("getTradingProperty", "canCreateStopLimit", instrument, Account);
        -- Check whether close market orders are allowed
        CanClose = core.host:execute("getTradingProperty", "canCreateMarketClose", instrument, Account);
        -- And finally turn "in lots" amount into the absolule value.
        Amount = instance.parameters.LotSize * core.host:execute("getTradingProperty", "baseUnitSize", instrument, Account);

    end
    -- name the indicator
    local name = profile:id() .. "(" .. instance.bid:name() .. "," .. instance.parameters.TF .. "," .. F .. "," .. S .. ")";
    instance:name(name);
    -- setup the signal. pay attention, we pass "ShowAlert" (value initially taken from the instance.parameters.ShowAlert)
    -- here, so, we don't check whether alerts are requested anymore.
    ExtSetupSignal(name, ShowAlert);
    -- and finally subscribe for the ticks of the instrument the user initially chosen to run the strategy for to
    -- have our strategy activated once.
    ExtSubscribe(1, nil, "t1", true, "tick");
end

-- the function is called every time when any subscribed price is changed. For tick subscribtions the function is called
-- for every tick, for the bar subscribtions the function is called when the candle is closed (in other words, when
-- the first tick of the next candle appears).
function ExtUpdate(id, source, period)
    if id == 1 and price == nil then
        -- our tick subscribtion. do it on the first tick if the
        -- user chosen subscribtion has not been not made yet.

        -- subscribe for the user chosen timeframe/to close prices
        price = ExtSubscribe(2, nil, instance.parameters.TF, true, "close");
        -- create indicators
        fastMA = core.indicators:create("MVA", price, instance.parameters.F);
        slowMA = core.indicators:create("MVA", price, instance.parameters.S);
        -- and get the oldest index of the bar we can work at
        first = math.max(fastMA.DATA:first(), slowMA.DATA:first()) + 1;
    elseif id == 2 then
        -- on the user chosen subscription (can be either tick or bar subscribtion).

        -- update indicators
        fastMA:update(core.UpdateLast);
        slowMA:update(core.UpdateLast);

        -- if we have enough bars in the history to work
        if period >= first then
            -- if fast moving average goes over slow moving average
            if core.crossesOver(fastMA.DATA, slowMA.DATA, period) then
                -- show the signal
                ExtSignal(instance.ask, instance.ask:size() - 1, "BUY", SoundFile, nil);
                -- and if the trading is allowed - either close all sell positions or
                -- open the buy one
                if CanTrade then
                    if not(close(true)) then
                        open(false);
                    end
                end

            -- if fast moving average goes under slow moving average
            elseif core.crossesUnder(fastMA.DATA, slowMA.DATA, period) then
                -- show the signal
                ExtSignal(instance.bid, instance.bid:size() - 1, "SELL", SoundFile, nil);
                -- and if the trading is allowed - either close all buy positions or
                -- open the sell one
                if CanTrade then
                    if not(close(false)) then
                        open(true);
                    end
                end
            end
        end
    end
end

-- The function opens the position in the chosen direction.
-- Account, amount and instrument (aka offer) are predefined in
-- the Prepare() function.
function open(sell)
    -- create and fill value map with the predefined order parameters
    local valuemap;
    valuemap = core.valuemap();
    valuemap.Command = "CreateOrder";       -- command: create new order
    valuemap.OrderType = "OM";              -- order type: open market (execute immediatelly, at any available market price)
    valuemap.OfferID = OfferID;             -- instrument
    valuemap.AcctID = Account;
    valuemap.Quantity = Amount;

    -- fill the side and calculate stops and limits.
    -- the Stop and Limit global variables are expressed in pips.
    -- pay attention that we calculate prices against different prices
    -- to avoid setting the stop or limit inside the spread.
    -- Also, the stop and limit offset depends on the trade direction. For example
    -- the stop for a buy position is below the price while the stop for a sell position
    -- is above the price.
    local side, limit, stop;
    if sell then
        side = "S";
        stop = instance.ask[instance.ask:size() - 1] + Stop * instance.ask:pipSize();
        limit = instance.bid[instance.ask:size() - 1] - Limit * instance.bid:pipSize();
    else
        side = "B";
        stop = instance.bid[instance.ask:size() - 1] - Stop * instance.ask:pipSize();
        limit = instance.ask[instance.ask:size() - 1] + Limit * instance.bid:pipSize();
    end

    -- fill side and stop/limit orders in the value map
    valuemap.BuySell = side;

    if Stop > 0 and CanStop then
        valuemap.RateStop = stop;
    end

    if Limit > 0 and CanStop then
        valuemap.RateLimit = limit;
    end

    -- and, finally, execute it
    local success, msg;
    success, msg = terminal:execute(200, valuemap);
    assert(success, msg);
end

-- close all the positions in the specified direction
function close(sell)
    local enum, row, valuemap, side, closed;
    local valuemap;

    -- just prepare the name of the direction ("S" for sell and "B" for buy).
    if sell then
        side = "S";
    else
        side = "B";
    end

    -- the flag indicating whether at least one position is closed.
    closed = false;

    -- get a enumerator for the "open trades" trading table
    enum = core.host:findTable("trades"):enumerator();

    -- and scan trough all table rows. one row is one trade.
    while true do
        row = enum:next();
        if row == nil then
            break;
        end

        -- if trade was placed for chosen account and instrument
        -- and the trade is in the requested side - close it
        if row.AccountID == Account and
           row.OfferID == OfferID and
           row.BS == side then

            -- use different close models depending on the account type
            if CanClose then
                -- non-NFA accounts, can use Market Close order
                valuemap = core.valuemap();
                valuemap.Command = "CreateOrder";
                valuemap.OrderType = "CM";      -- close market order
                valuemap.OfferID = OfferID;
                valuemap.AcctID = Account;
                valuemap.Quantity = row.Lot;    -- the size is required because we can close the position partially
                valuemap.TradeID = row.TradeID; -- the trade must be referenced int he close order
                -- the side of the close order must be opposite to the side of the trade.
                if row.BS == "B" then
                    valuemap.BuySell = "S";
                else
                    valuemap.BuySell = "B";
                end
                -- and execute
                local success, msg;
                success, msg = terminal:execute(200, valuemap);
                assert(success, msg);
            else
                -- NFA accounts. We cannot call close orders but can
                -- execute an opposite open order. Since NFA accounts can
                -- never be hedging accounts, the opposite open order
                -- closes the existing positions first.
                valuemap = core.valuemap();
                valuemap.OrderType = "OM";
                valuemap.Command = "CreateOrder";
                valuemap.OfferID = OfferID;
                valuemap.AcctID = Account;
                valuemap.Quantity = row.Lot;
                if row.BS == "B" then
                    valuemap.BuySell = "S";
                else
                    valuemap.BuySell = "B";
                end
                local success, msg;
                success, msg = terminal:execute(200, valuemap);
                assert(success, msg);
            end
            -- we closed the position, so we should not open anything.
            closed = true;
        end
    end
    return closed;
end

-- use the helpers
dofile(core.app_path() .. "\\strategies\\standard\\include\\helper.lua");

Step 2: Debugging, Part 1
Step 2. Debugging Strategy

If you did everything right, the final code of the strategy is located in the “C:\Program Files\Gehtsoft\IndicoreSDK\strategies\” folder in the sample.lua file. Now let’s look how it works.

At the first step we must prepare two price histories to test the strategy at. The first price history will be used to simulate ticks. The debugger has four models to simulate the ticks using historical bars:

a) one tick per bar using close prices.
b) three ticks per bar, always high, then low, then close price of the bar
c) three ticks per bar, always low, then high, then close price of the bar
d) “Mixed mode”. The option (b) or (c) is chosen for each bar individually. The ascending bar goes using option (c), the descending bar goes using option (b).

The second collection is for simulating the user chosen timeframe. Let’s simulate ticks using 1-minute candles and test our strategy on 15-minutes candles.

To prepare the data just open both timeframes in Marketscope for the same instrument and approximately the same time period and then save both of them using “File->Export to Indicore...” command of Marketscope. The best location to save the data is the default SDK’s data folder: “C:\Program Files\Gehtsoft\IndicoreSDK\data\”. Name the files so you can easily find them in future, let’s use “MY-EURUSD-1m.csv” and “MY-EURUSD-15m.csv” for our example.


download the data(below).

Now run the strategy debugger (“Start->Programs->IndicoreSDK->Lua Strategy Debugger”). In the debugger window, choose “File->Open Strategy”. The debugger shows the list of the strategies which can be debugged. Now we have only one, our first strategy, so only one choice is available. Click “Ok” to start the strategy.


The strategy is opened. In case any error appears, it will be shown in the output window. If there are no errors in the strategy, it will be loaded and execution will be suspended right before the first line of the strategy. You can now execute the strategy step by step or between break points.



Let’s skip the whole initialization routine. To do that, just scroll down to the first statement in the Prepare() function, put the cursor at the line with the first statement and then press F9 to put the break point.

Debugging: Part 2

Now just press F5 (run) to execute the Init() function. When the Init() function is finished, the prompt for the strategy parameter appears.



Choose the previously prepared 1-minute data for the simulating ticks (MY-EURUSD-1m.csv) collection. Then choose the simulation mode to mixed (so we will have 3 ticks per each 1 minute candle). Then scroll the list of the parameters to the “Time Frame” parameters and enter “m15” (15 minutes) timeframe to apply our strategy on 15-minutes data. Press “Set” to setup the parameter value. Press OK to setup the indicator parameters and go to the “Prepare” function.

Now look at another useful function of the debugger. You can check the value of any Lua expression anytime. Switch to the “Watches” tab and then use “Debug->Add Watch” command to add a Lua expression and see the value. All expressions entered into watches list are updated after every step, so you can monitor the status of your strategy anytime during the execution.



Ok. Let’s go ahead. Now try to stop at the following moments: when the chosen price history is requested and when open or close functions are called.




Then Run (F5) the code again. The debugger stops at the first simulated tick. Now you can look at the “Stream” output window. You can see simulated bid and ask ticks and the report about your strategy execution, where all the strategy alerts and trading operations and their results (long and short amount, floating P/L, and balance) are shown.



Now press F8 to perform the subscription. The request for the file with data for our subscription appears. Choose the prepared 15-minutes snapshot (MY-EURUSD-15m.csv) file and press OK.



Now run the strategy again until another breakpoint (on short or long signal) appears. Then go step-by-step until the CreateOrder command is executed. You see that the short amount appeared in the “streams” window. You can also switch to the “trades” to see the trade table, where a trade appears too. Now the trade will be changed every time a new tick appears to check stop and limit orders (if any are set up) and to recalculate P/L. The “streams” window as well as “accounts” and “summary” tables will be changed accordingly.

Note: there is a small difference with the actual execution of the orders in the Trade Station. While the debugger and back testing reflects the order in the tables immediately, in real life it takes 20-50 milliseconds before the trade is actually executed and up to 500-750 milliseconds while the trade appears in the table. In most cases, it is not important for most of strategies, but never check the trade immediately after the terminal:execute() command. The second difference is that while in real life any order can be filled partially (i.e. can be executed in full but produce a number of trades, or produce a trade for partial amount), the debugger always executes the order in full.



Now you can go either step-by-step (F8) or run between points until the strategy is completely simulated on the chosen data. You can monitor the execution log (“Streams”) as well as the trading tables changes on every step.


When the strategy is finished, the “update is finished” message appears in the output window. You can check the whole history of the execution in the “Streams” window and/or the log of the trading in “ClosedTrades” window.

Step 3. Backtesting
Well, now we’re sure that the strategy works exactly as you wanted. Now you can try it under Trading Station in backtesting mode. The backtesting uses exactly the same simulation routine as the debugger. The only difference is that you don’t need to save data, the backtesting takes data to simulate ticks directly from the chosen chart and loads all the additional data from the price server.

So, first install the strategy into Trading Station. The simplest way to do it is just copy the sample.lua file from the “C:\Program Files\Gehtsoft\IndicoreSDK\strategies\” folder into “C:\Program Files\CandleWorks\FXTS2\Strategies\Custom\” folder. (Note: Copy, don’t move! If you will delete the strategy using “Custom Strategy Manage” function in Trading Station, your strategy will be removed from the disk!).

Now, let’s backtest the indicator using default settings (with no stop and limit orders). In that case we can backtest right using the timeframe we plan to apply the strategy to, because the sell and buy conditions are checked only when the bar is just closed. So, open 15-minutes chart of EUR/USD and load as much data as you want to backtest. Then choose “Strategies->Backtest” command. In the form that appears, click on ellipsis button near the name of the strategy to backtest and in the strategy form choose our strategy (SAMPLE). Leave the strategy parameters unchanged and set the “Set period of the signal to the chart” to “Yes”. Using the parameter, the backtester automatically sets the “Time Frame” parameter to the timeframe of the chart (15 minutes in our case). Also, please, set the account mode to the “Non-hedging account” to let the strategy use the stop and limit orders as well as close orders.



Now apply the backtesting.

Now you can see all the alerts and orders which are executed by the strategy as well as the equity curve. It’s not quite profitable, huh? Well it is not intended to be profitable. Just to be simple. You can also see the report. Just right-click on the equity curve area and choose “Show Report” menu item. The report is opened in the browser.



Well, now make the task a bit harder. Let’s start using the stop and limit orders to manage the risk. In that case, because stop and limits monitor the current market, we must simulate as many ticks as we can. So, choose the 1 minute time frame, so we can simulate 3 ticks per a minute. Not wonderful, but good enough in most cases.

Then change the backtest parameters the following:

1) Open the backtest parameters (right click on “BACKTEST(SAMPLE)” label and choose “Change indicator BACKTEST”).
2) Set the “Set period of the signal to the chart” to “No”, so now the minute (currently opened) chart will be used for the tick simulation and the timeframe chosen in the strategy parameters will be additionally loaded.
3) Open the strategy properties (right-click on the ellipsis button near the “signal name” parameter) and set up the following:
a) The timeframe - to 15 minutes
b) The stop to 15 points
c) The limit to 30 points
d) Set the “show alert” to “no” to hide alerts from backtesting



Now apply the new parameters. You can also download and install MF_MA indicator (the indicator which shows the higher time frame moving average), so you can check whether conditions are checked properly.

This is very good thread. Wanted to start new thread but I do not have enough posts. Hope someone will see my question about debugging.

the problem:
I started creating strategy that will start to trade immediately. That strategy should get all data from history, calculate whatever needed and start trading when first opportunity arises.
(To be precise, I wanted to take 300 H1 data and 300 D1 data. Then i would find all H1 data from 17h previous day until now and calculate one part of the logic. Then I would use several D1 days data to calculate another part of the logic)

Help says:
ExtSubscribe will return 300 bars/ticks back from now.

But, when I ask for D1 data I get 300 bars. (not a problem)
When I ask for H1 data (this is time frame for the strategy) I get only 2 bars of data.

Is this ok, or am I wrong?

My logic works in debugger because I use data that suits me:
For testing I should use this dates. Imagine that today is February 3, 2011.
For source, I would use from February 1, 2011. to February 3.
For history (D1 or H1 or any time frame) 300 bars before February 1, 2011.

Although this works in debugger with this data, it does not work in the backtest.

So, any suggestions? Link to a specification will be just fine. I am struggling with this for some time now....

Thanks!

Hello vlada.pa!

It would be good if you can provide a code for downloading data.

Hello Timon55,

Thank you for reply!

I did not use any special code for downloading data. Only ExtSubscribe, which is inside help.lua

Here is fragment of my code. Example that can be open in debugger is in the attachment.

Code:

function Prepare()
 ...
    ExtSubscribe(1, nil, "t1", true, "tick");    

    loaded = false;    
    loadedDayPrice = false;    
    init = false;
 ...
end

function ExtUpdate(id, source, period)
    if init == false then
        -- subscribe for the user chosen timeframe/to close prices
        if price == nil then
            price = ExtSubscribe(2, nil, instance.parameters.TF, true, "bar");
        end
        
        if dayHistoryPrice == nil then
            dayHistoryPrice = ExtSubscribe(3, nil, "D1", true, "bar");
        end        
        
        init = true;    
    end

    if id == 3 then
        loadedDayPrice = true;    
    end

    if id == 2 then
        -- inside user chosen subscription
        
        -- wait to have dayPriceHistory and price
        if loadedDayPrice == false then 
            return;
        end

        -- HERE I ASSUME THAT ALL DATA IS LOADED. 

        -- this will be done only once        
        if loaded == false then
            
            -- find time when a day was finished.
            local dayoffset = core.host:execute("getTradingDayOffset"); 
            local hourOffset = (0.5/12)*dayoffset; -- -7Hours
            if hourOffset < 0 then
                hourOffset = 1 + hourOffset;
            end

            -- find time when last day started
            local todayStartTime;
            -- hourOffset <= last price bar < 1
            if math.fmod(price:date(price:size()-1), 1) >= hourOffset then 
                todayStartTime = math.floor(price:date(price:size()-1)) + hourOffset;
            else -- 0 < last price bar < hourOffset
                todayStartTime = math.floor(price:date(price:size()-1)) + hourOffset -1;
            end
            
            
            -- go from the last price in price table and go toward first. when you find bar that is before todayStart then you will have correct todayHigh and todayLow
            local priceCnt = 1;            
            while price:hasData(price:size()-priceCnt) and price:date(price:size()-priceCnt) >= todayStartTime do
                if todayHigh == 0 or price.high[price:size()-priceCnt] > todayHigh then
                    todayHigh = price.high[price:size()-priceCnt];
                end
                if todayLow == 0 or price.low[price:size()-priceCnt] < todayLow then
                    todayLow = price.low[price:size()-priceCnt];
                end
                priceCnt = priceCnt + 1;
    
                -- *********** PROBLEM IS HERE. 
                -- there are no 300 bars
                assert( price:hasData(price:size()-priceCnt) );
            end
           

            -- now find last day from day history
            local indexOfTheYesterday = core.findDate(dayHistoryPrice, price:date(price:size()-priceCnt), false);
            
            yestHigh = dayHistoryPrice.high[indexOfTheYesterday];
            yestLow = dayHistoryPrice.low[indexOfTheYesterday];
            yestClose = dayHistoryPrice.close[indexOfTheYesterday];

            assert( indexOfTheYesterday - 1 >=0, "indexOfTheYesterday < 0" );
            assert( dayHistoryPrice:hasData(indexOfTheYesterday), "No data for day.");
            assert( dayHistoryPrice:hasData(indexOfTheYesterday-1), "No data for day.");
      

            ExtSignal(instance.bid, instance.bid:size() - 1, "LOADED", nil, nil);
            loaded = true;
        end       
    
        -- if we have enough bars in the history to work
        if loaded and period >= first and source:hasData(period) then
             
        end
    end
end

....

I used:
test_eurusd-h1.csv as source and H1 data,
test_eurusd-d1.csv as D1 data
and got error. I think same thing happens in BACKTEST.

When I used
test_eurusd-h1_source.csv (begins several days after test_eurusd-h1.csv) as source,
test_eurusd-h1.csv as H1 data,
test_eurusd-d1.csv as D1 data
it worked ok.

Attachment contains lua file and csv files mentioned above.

All suggestions are welcome.

Regards, Vlada

Hello vlada!

To solve your problem just move price subscribing to the Prepare() function and remove all checkings for data loading from the Update() function.

For example:

Code:

function Prepare()
    ExtSubscribe(1, nil, "t1", true, "tick");    
   
    if price == nil then
        price = ExtSubscribe(2, nil, "H1", true, "bar");
    end
    
    if dayHistoryPrice == nil then
        dayHistoryPrice = ExtSubscribe(3, nil, "D1", true, "bar");
    end
end

function ExtUpdate(id, source, period)

    core.host:trace("ID = "..id.." period = "..period);
    -- find time when a day was finished.
    local dayoffset = core.host:execute("getTradingDayOffset"); 
    local hourOffset = (0.5/12)*dayoffset; -- -7Hours
    if hourOffset < 0 then
        hourOffset = 1 + hourOffset;
    end
    
    -- find time when last day started
    local todayStartTime;
    -- hourOffset <= last price bar < 1
    if math.fmod(price:date(price:size()-1), 1) >= hourOffset then 
        todayStartTime = math.floor(price:date(price:size()-1)) + hourOffset;
    else -- 0 < last price bar < hourOffset
        todayStartTime = math.floor(price:date(price:size()-1)) + hourOffset -1;
    end
            
            
    -- go from the last price in price table and go toward first. when you find bar that is before todayStart then you will have correct todayHigh and todayLow
    local priceCnt = 1;            
    while price:hasData(price:size()-priceCnt) and price:date(price:size()-priceCnt) >= todayStartTime do
        if todayHigh == 0 or price.high[price:size()-priceCnt] > todayHigh then
            todayHigh = price.high[price:size()-priceCnt];
        end
        if todayLow == 0 or price.low[price:size()-priceCnt] < todayLow then
            todayLow = price.low[price:size()-priceCnt];
        end
        priceCnt = priceCnt + 1;
    
                -- *********** PROBLEM IS HERE. 
                -- there are no 300 bars
        core.host:trace(price:size());
        assert( price:hasData(price:size()-priceCnt) );
    end
           

            -- now find last day from day history
    local indexOfTheYesterday = core.findDate(dayHistoryPrice, price:date(price:size()-priceCnt), false);        
    yestHigh = dayHistoryPrice.high[indexOfTheYesterday];
    yestLow = dayHistoryPrice.low[indexOfTheYesterday];
    yestClose = dayHistoryPrice.close[indexOfTheYesterday];
    assert( indexOfTheYesterday - 1 >=0, "indexOfTheYesterday < 0" );
    assert( dayHistoryPrice:hasData(indexOfTheYesterday), "No data for day.");
    assert( dayHistoryPrice:hasData(indexOfTheYesterday-1), "No data for day.");
      

    ExtSignal(instance.bid, instance.bid:size() - 1, "LOADED", nil, nil);
    loaded = true;
    
    -- if we have enough bars in the history to work
    if loaded and period >= first and source:hasData(period) then
             
    end
end

Your suggestion was very helpful. Thank you!
Regards, Vlada

Hi
I am an experienced programmer but totally new to indicore SDK

i am trying to develop a RSI alert which reports immediately when the RSI crosses a level and not at the end/start of a new candle as i want to use it primarily on a H1 timeframe and if it crosses say 5 minutes after the start it is not much use if it only reports it at the start of the next candle

i used a RSI strategy sample and in the Prepare function it has the following :
gSource = ExtSubscribe(1, nil, instance.parameters.Period, instance.parameters.Type == "Bid", "close");

RSI = core.indicators:create("RSI", gSource, RSIN);

in the function ExtUpdate(id, Source, Period)
it has
RSI:update(core.UpdateLast);
if not(RSI.DATA:hasData(period - 1)) then
return ;
end

Would it be possible to tell me what i must change so that it checks every price change and not every new candle
it would be very much appreciated
sorry for asking something which is obviously so elementary but i have hit a brick wall
i am attaching the ful .lua in case you want to look at it

Hi Strauss.c.m,

Yuo need additionally subscribe to ticks. Below is a sample of code how to do this:

Code:

tickSource = ExtSubscribe(2, nil, "t1", instance.parameters.Type == "Bid", "close");
...
function ExtUpdate(id, Source, Period)
   if id == 2 then
       -- called every tick
   else
       -- called every hour in your case
   end

Best regards,
Ekaterina

Hi Ekaterina
Thanks very much for your reply - i now get the extupdate firing every tick but the data seems to be corrupt as it is reporting crosses that are not happening
i added a source for the tick data and also created a RSI with the tickdata source
when i run it in the debugger i can see it appears to be getting bad data
sorry about asking again but it is clear that there is something i do not understand
i have attached my modified lua file
i really would appreciate it if you could give me a pointer as to what i am getting wrong
thanks
Christopher

CMS_RSI.lua
You should build RSI on candles but check it on tick source update.
But there is one thing: when extupdate is called with id=2 (tick in your case) period is valid for the tick source. In order to check the last value of RSI based on candles you need to access it with different period: take size() - 1 of the RSI_Candle.DATA stream. Like this:

Code:

RSI_Candle = core.indicators:create("RSI", gSource, RSIN);
...
if id == 2 then
   -- you can't use the "period" to acces RSI_Candle. It's valid for different source
   -- use RSI_Candle.DATA:size() - 1 as the last period of RSI_Candle
   if core.crossesOver(RSI_Candle.DATA, OB, RSI_Candle.DATA:size() - 1) then
...

Hi sibvic and Ekaterina
The Alert now functions correctly and i just want to say thank you so much for your assistance
I have now found that sometimes the price will spike through a limit and then withdraw and go the other way and core.crosses.... function picks up the cross but does not pick up the withdrawal. can you please tell me where i can find the core functions and the parameters to address them and the properties for RSI, MACD and MVA
Thanks again so much